Adding Checksums to Golang Webdav Server Part II

Continuing from previous post, this part deal with how to integrate rclone handling to an actual webdav server.

What Previous Post didn’t Do?

Previous post deals with how rclone interacts with OwnCloud and NextCloud webdav servers and the implementation of custom webdav filesystems. It doesn’t cover how to extract the relevant headers and write these information to checksum files.

The Problem with Golang Webdav Handler

Golang webdav handler can only be customized by implementing custom LockSystem and FileSystem. We can not modify how webdav handler handle incoming requests, such as tell a client to redirect to another server to download the file or run specific operations after a request is fulfilled.

The first is useful when serving files not hosted on that server and traffic will be redirected to the server actually serving the files. The second is useful to track the status of file operations like what was uploaded and emit detailed logs. Cloudreve solves these problems by using modified webdav source code, and it solves some of the problems.

The second part can be kind of tackled by tracking how filesystem open files and record relevant information in the file struct, but that’s too much trouble and error-prone.

Solution

Because we’re interested in adding checksum support to webdav, we can instead track the status of http.ResponseWriter, as after webdav handler has done its handling, the status can give us a glimpse of whether the request is successful.

Wrapping ResponseWriter

The wrapped response writer should let us know whether the request succeeds, as go http will implicitly write 200 status when starting writing to the writer, we consider a status of 0 as a success.

type responseWriterWrapper struct {
    http.ResponseWriter
	status int
}

func (w *responseWriterWrapper) WriteHeader(status int) {
	w.status = status
	w.ResponseWriter.WriteHeader(status)
}

func (w *responseWriterWrapper) IsSuccessful() bool {
	return w.status == 0 || (w.status >= 200 && w.status <= 299)
}

Request Postprocessing

From previous post we know that rclone only attaches modified time information with COPY, MOVE and PUT requests, and checksum information with PUT requests, and we can change the modification time of the file or write checksums in the postprocess step.

dw := &davWriter{writer, 0}
handler.ServeHTTP(dw, request)

switch request.Method {
case "COPY", "MOVE":
    if !dw.IsSuccessful() {
        break
    }

    modtime := parseOCMtime(request)
    if modtime.IsZero() {
        return
    }

    dst, _ := url.Parse(request.Header.Get("Destination"))
    updateModtime(stripPrefix(dst.Path), modtime)
case "PUT":
    if !dw.IsSuccessful() {
        break
    }

    modtime := parseOCMtime(request)
    if !modtime.IsZero() {
        updateModtime(stripPrefix(request.URL.Path), modtime)
    }
    checksum := request.Header.Get("OC-Checksum")
    if checksum != "" {
        updateChecksum(stripPrefix(request.URL.Path), checksum)
    }
}

parseOCMtime is a util function that tries to parse the request’s X-OC-Mtime header. The header, if present, will contain the unix time as a string. stripPrefix is a util function that mirrors what golang webdav handler does to extract the file path. Because COPY and MOVE are targeted at the destination, not the source, we instead use destination path to update the modified time.

Limitations

The method used is susceptible to race condition, as when webdav handler is done, relevant locks may also be released, to solve this problem we need to modify the source of webdav like cloudreve did. Existing files also don’t have checksums attached to them. We can use rclone itself to calculate the hashes and write them to the appropriate paths though. We also relied on the request to provide the checksum information, which we can calculate when the request body is being read. That part is left as an exercise.