Originally TextMaster had limit of 2.4 GB for the text file size. At the time text file of that size would be considered too big to be real. The following e-mail changed that:
“Hey Jake,
We do not have the file merged yet, but I need to QC the merged file and no way to actually open it, there will be about 150K records in it and length will be around 20000.
Paul”
150,000 records in text file is not unusual but record length of 20 K is very unusual. All together, file size would be 3 GB. Because file wasn’t available I had to create it. Data generator did it within a few minutes and I had 150 K lines fix length file with 20 K record length. I created connection (file description) which had two fields with length of 10 K each.
Query “Select top(10) * from C:\TMSampleData\HugeFile.tmc “ started and reported an error but execution continued. I got first ten lines from my HugeFile. I was impressed. When I tried to select all records from the file error message appeared but process continued. Little bit of debugging pointed to the fact that progress bar control can accept only 32 bit integer. Divide by 1000 for file size above 2 GB fixed the problem. It was released in August but blog wasn’t updated. Sorry.
Now limit is 300 GB. Is it enough? Only time will tell.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment