Downloaded files changed

[GrimPixel]

There is my repo: https://codeberg.org/GrimPixel/Public_Diagrams
This script downloads images from Wikimedia and organise them in directories by author name.

The problem is, some files downloaded are different from what they are supposed to be. For example, A7N8X/PowerHead_Polini.svg is normal, A7N8X/Testata_Mototekna.svg is failed by having additional characters “뿯붿”.

[ketpatil77]

For image/SVG downloads, preserve bytes rather than decoding through response.text and writing text again. This minimal helper keeps binary content intact:

def save_response(client, url, destination):
    with client.stream("GET", url) as response:
        response.raise_for_status()
        with open(destination, "wb") as output:
            for chunk in response.iter_bytes():
                output.write(chunk)

iter_bytes() applies HTTP Content-Encoding decompression, which is normally what you want for the usable downloaded file. iter_raw() instead preserves the compressed transfer payload, so it is not interchangeable when Content-Encoding is gzip/br/etc.

The characters alone do not establish where corruption occurred. Compare a hash of the bytes actually saved, inspect Content-Type/Content-Encoding, and check the first bytes of the file; an HTML error page or a text decode/re-encode step can both explain a different result. An SVG is normally XML text, but downloading it as bytes preserves its declared encoding. I have not inspected or reproduced the linked Codeberg project, so this is a diagnostic path rather than a confirmed fix for that script.

Validated exact saved bytes for a gzip-encoded binary response, including non-UTF-8 bytes, through MockTransport. Binary content, streaming.

Validation version: HTTPX 0.28.1, Python 3.11.9.