Is There A Python Module For Creating Test Files Of Specific Sizes?

I have a server that has files uploaded to it. I need to profile the upload/response time for various file sizes to that server i.e. how long it takes to upload a 10kb file, a 100m

Solution 1:

You don't really need to write 1MB to create a 1MB file:

withopen('bigfile', 'wb') as bigfile:
    bigfile.seek(1048575)
    bigfile.write('0')

On the other hand, do you really need a file at all? Many APIs take any "file-like object". It's not always clear whether that means read, read and seek, iterating by lines, or something else… but whatever it is, you should be able to simulate a 1MB file without creating more data than a single read or readline at a time.

PS, if you're not actually sending the files from Python, just creating them to use later, there are tools that are specifically designed for this kind of thing:

dd bs=1024 seek=1024 count=0 if=/dev/null of=bigfile # 1MB uninitializeddd bs=1024 count=1024 if=/dev/zero of=bigfile # 1MB of zeroesdd bs=1024 count=1024 if=/dev/random of=bigfile # 1MB of random data

Solution 2:

Just do:

size = 1000withopen("myTestFile.txt", "wb") as f:
    f.write(" " * size)

Solution 3:

I'd probably use something like

with tempfile.NamedTemporaryFile() as h:
    h.write("0" * 1048576)
    # Do whatever you need to do while the context manager keeps the file open# Once you "outdent" the file will be closed and deleted.

This uses Python's tempfile module.

I used a NamedTemporaryFile in case you need external access to it, otherwise a tempfile.TemporaryFile would suffice.

Solution 4:

From @abarnert and @jedwards answers:

mb = 1with tempfile.TemporaryFile() as tf:
    tf.seek(mb * 1024 * 1024 - 1)
    tf.write(b'0')
    tf.seek(0)