How to split large file on Windows?
Is there a tool to split large text file (9Gb) into smaller files so that I can open it and look through?
Anything usable from command line that comes with Windows (XP)?
Or what's the best way to split it? Can I use 7z to create separate volumes and then unzip one of them separately? Will it be readable or does it need all the other parts to unzip into the big file again?
Update
I put together quick 48 lines python script that split the large file into 0.5GB files which are easy to open even in vim. I've just needed to look through data towards the last part of the log (yes it is a log file). Each record is split across multiple lines so grep would not do.
39 Answers
There is a freeware Windows file splitter called HJSplit
Available here. The website claims it can split files of any type and size, but 9GB is a big file.
7The GNU Core Utils package (available here for Windows) includes the Split utility.
The --help documentation is as follows:
Usage: split [OPTION] [INPUT [PREFIX]]
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default
size is 1000 lines, and default PREFIX is `x'. With no INPUT, or when INPUT
is -, read standard input.
Mandatory arguments to long options are mandatory for short options too. -a, --suffix-length=N use suffixes of length N (default 2) -b, --bytes=SIZE put SIZE bytes per output file -C, --line-bytes=SIZE put at most SIZE bytes of lines per output file -d, --numeric-suffixes use numeric suffixes instead of alphabetic -l, --lines=NUMBER put NUMBER lines per output file --verbose print a diagnostic to standard error just before each output file is opened --help display this help and exit --version output version information and exit
SIZE may have a multiplier suffix: b for 512, k for 1K, m for 1 Meg.For example, to split input.txt into 100Mb chunks, only splitting at the ends of lines,
split input.txt -C 100mwill give you output files named xaa, xab, xac, etc.
2One can use 7-Zip to create segments of text file in certain size (e.g. 100 MB segments out of 1.5 GB log file).
The Key options are - Use "Store" as opposed to "Compress" - Use "Split to volumes"
You should be able to see text in .001 (.nnn) files.
Another is GSplit - according to their site it can split very large files (larger than 4Gb <-- since they crossed the 4Gb limit, I guess they can do 9 Gb as well).
But, another thing - you say you want to split it into smaller parts so you can open it up and look at it. That sounds like a very big perhaps log file.
In any case, for opening large text files, may I recommend EmEditor - they claim themselves it can open very large files (up to cca. 250 Gb), and I've used it in the past for files up to 2 Gb. But in any case, I think it may be a better solution than splitting.
3Check out Large Text File Viewer, it's great for things like this. Most archivers and splitters will separate the file into pieces which cannot be used to read each piece of data independently and properly, you need to extract them all to get the file back.
Large Text File Viewer is free and portable.
6You can use 7zip itself to split the files. (You can save as a .zip or .7z format.) When you go to create the archive there is an option called "Split volume, bytes". Just select how large you want the chunks.
And yes, you can unzip them individually if you wish.
There's an online tool that splits text files if anyone is looking to split files quickly. .
Works great for me. And splits files respecting lines which is what I was looking for. It also says it's all HTML5 client side so it's safe to use. I'm not sure how big it can go but I think it depends on your machine's ram.
4The idea of seeing part of the file before deciding what to do with it is for me the best option.
The Large Text Viewer App can be installed on Windows through the Microsoft Store and it offers an option to cut the file in chunks of size. It may well be that it uses the same editor previously mentioned (behind the scenes), but the option to install it from a known source is better IMHO than the alternative links offered. It worked great for me.
The only issue if splitting by size is that it not necessarily breaks the file at a good place, so you may have to edit it to capture the exact content you want.
1Splitting files is also a funtion of Total Commander, the tool I can't do my work without. Get your 30-day trial here: Licences are dirt-cheap, concurrent and permanent.
In Total Commander, highlight te file you want to split. Select [file][split file] from the menu. In the pop-up select your target-directory and "bytes per file". Choose from: 1.44 MB, 1.2 MB, 720K, 360 k, 100 MB, 250 MB, 650 MB or 700 MB. Press [OK] and watch the magic happen...
1
