WebFlux File Compression and Download Implementation

1. Introduction

Implementing such a feature under the traditional SpringMVC framework isn't difficult, but in reactive programming with WebFlux, it's quite different - you need to be non-blocking and conform to reactive specifications. Let's take a look.

2. Starting from the Entry Point - Frontend

Let's assume a common scenario where users need a batch export feature, so they check multiple items in a form, get the corresponding IDs, and pass them to the backend. Here I'm passing the filenames directly.

 		/**
     * Batch export
     */
    function batchExport(){

        // Get the form
        const form = document.getElementById("fileForm");

        // Get all checked checkboxes
        const checkedBoxes = [];
        const checkboxes = form.querySelectorAll('input[type="checkbox"]');
        checkboxes.forEach((checkbox) => {
            if (checkbox.checked) {
                checkedBoxes.push($(checkbox).attr("id"));
            }
        });

        // Async request
        fetch(contextPath + "share/batchDownloadFile", {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json'
            },
            body: JSON.stringify({
                fileNames: checkedBoxes,
            })
        })
        .then(response => {

            const disposition = response.headers.get('Content-Disposition');
            const filename = disposition ? disposition.split('filename=')[1].replaceAll("\"","") : 'compressedFiles.zip';
            return response.blob().then(blob => {
                const url = window.URL.createObjectURL(blob);
                const a = document.createElement('a');
                a.href = url;
                a.download = filename;
                document.body.appendChild(a);
                a.click();
                window.URL.revokeObjectURL(url);
            });
        })
        .catch(error => {console.error('Error:', error);});
    }

Since we're sending a POST request, the backend returns a byte array directly. We need to convert the returned data and provide a virtual anchor tag to trigger the browser download.

const disposition = response.headers.get('Content-Disposition');
const filename = disposition ? disposition.split('filename=')[1].replaceAll("\"","") : 'compressedFiles.zip';
return response.blob().then(blob => {
    const url = window.URL.createObjectURL(blob);
    const a = document.createElement('a');
    a.href = url;
    a.download = filename;
    document.body.appendChild(a);
    a.click();
    window.URL.revokeObjectURL(url);
});

Now let's look at the backend part.

3. Data Processing - Backend

Use Mono<String> requestBody directly to receive frontend request parameters. Note that you shouldn't do this like in traditional MVC frameworks, where you'd write on the first line:

log.info("batchDownloadFile received payload:{}", requestBody);

This would be problematic because requestBody isn't the actual request parameter, but a Mono object.

Complete code:

/**
 * Batch download files
 *
 * @param requestBody request body
 * @return {@link Mono}<{@link ResponseEntity}<{@link Flux}<{@link DataBuffer}>>>
 */
@PostMapping(value = "/batchDownloadFile", produces = "application/zip")
public Mono<ResponseEntity<Flux<DataBuffer>>> batchDownloadFile(@RequestBody Mono<String> requestBody) {
    return requestBody.doOnNext(it -> log.info("batchDownloadFile received payload:{}", it))
            .map(JSON::parseObject)
            .doOnError(it -> {
                it.printStackTrace();
                log.error("batchDownloadFile failed");
            })
            .map(it -> {
                // Get all filenames from input, iterate, and add to a zip file
                final Stream<String> fileNames = Optional.ofNullable(it.getJSONArray("fileNames"))
                        .orElse(new JSONArray())
                        .stream()
                        .map(fileName -> exportPath + File.separator + fileName);
                // Compress multiple files and convert to byte array
                return zipFileToByteArray(fileNames);
            })
            .flatMap(data -> {
                // The previous map passes a byte array here. We just return the byte array directly

                // Set response headers
                HttpHeaders httpHeaders = new HttpHeaders();
                httpHeaders.setContentDispositionFormData("Content-Disposition", "download.zip");

                final ResponseEntity<Flux<DataBuffer>> body = ResponseEntity.ok()
                        .headers(httpHeaders)
                        .contentType(MediaType.APPLICATION_OCTET_STREAM)
                        // Finally, the body in ResponseEntity is wrapped with new DefaultDataBufferFactory().wrap
                        .body(Flux.just(data).map(b -> new DefaultDataBufferFactory().wrap(b)));
                return Mono.just(body);
            });
}

/**
 * Compress multiple files and convert to byte array
 *
 * @param filepathList file path list
 * @return {@link byte[]}
 */
@SneakyThrows
private static byte[] zipFileToByteArray(Stream<String> filepathList) {
    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
    ZipOutputStream zipOutputStream = new ZipOutputStream(byteArrayOutputStream);
    filepathList.forEach(file -> addToZipFile(file, zipOutputStream));
    zipOutputStream.close();
    return byteArrayOutputStream.toByteArray();
}

/**
 * Add a file to the returned zip file
 *
 * @param fileName file name
 * @param zos ZipOutputStream
 */
@SneakyThrows
private static void addToZipFile(String fileName, ZipOutputStream zos){
    File file = new File(fileName);
    try (FileInputStream fis = new FileInputStream(file);){
        // The fileName here can control the nesting hierarchy of files in the generated zip
        ZipEntry zipEntry = new ZipEntry(FilenameUtils.getName(fileName));
        zos.putNextEntry(zipEntry);

        byte[] bytes = new byte[1024];
        int length;
        while ((length = fis.read(bytes)) >= 0) {
            zos.write(bytes, 0, length);
        }

        zos.closeEntry();
    }

}

3.1 Why is the response object Mono<ResponseEntity<Flux<DataBuffer>>>?

Some might wonder if so many nested layers are necessary.

Actually, just ResponseEntity<Flux<DataBuffer>> would work too, but if you write it that way, it's the same as traditional MVC. We know that in reactive programming, the entire stream only executes when subscribe is called, then elements are generated. So we need to return Mono<ResponseEntity> instead of ResponseEntity<T>.

Then ResponseEntity<Flux<DataBuffer>> - is another layer of wrapping necessary?

Using ResponseEntity<Flux<DataBuffer>> to wrap Flux<DataBuffer> is because when handling streaming data, Flux is typically used to represent a series of data chunks. This approach can asynchronously send data chunks to the client one by one, rather than waiting for all response data to be ready before sending. This asynchronous processing can improve performance and reduce memory consumption.