Login  Register

Re: /framework/datafile caching.

Posted by David E. Jones-2 on Mar 16, 2010; 1:48am
URL: http://ofbiz.116.s1.nabble.com/framework-datafile-caching-tp1594248p1594370.html


Sorry, that's silly. A bug can't exist without a requirement. Where was this requirement ever established? What in the design implies that this was a requirement? The API very clearly represents a process that reads the entire file into memory.

It sounds like it doesn't meet a requirement that you came up with. That isn't the definition of a "bug", well except for the purposes of trolling on mailing lists I suppose.

-David


On Mar 15, 2010, at 7:16 PM, BJ Freeman wrote:

> when code will not handle real world data, it is broken.
> you and I discussed this when importing xml file and I choose to parse
> them manually instead of using DOM.
> So what term is there besides bug for broken design.
>
> =========================
> BJ Freeman
> http://bjfreeman.elance.com
> Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=93>
> Specialtymarket.com <http://www.specialtymarket.com/>
>
> Systems Integrator-- Glad to Assist
>
> Chat  Y! messenger: bjfr33man
> Linkedin
> <http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>
>
>
> David E Jones sent the following on 3/15/2010 6:05 PM:
>> On Mar 15, 2010, at 6:58 PM, Adam Heath wrote:
>>
>>> BJ Freeman wrote:
>>>> one of my smaller import files (8mb) is taking forever to be read in and
>>>> there is no output.
>>>> I am seeing the memory rail against the max setting.
>>>> got this error.
>>> The datafile set of classes is very broken for large files.  It has a
>>> List<Record>, which means it will copy the entire file into memory
>>> before doing anything with it.  The datafile code is not designed to
>>> handle large files.
>>>
>>> I consider this a bug that needs to be fixed.  Checking...
>>
>> This should be possible, but may require API changes. The problem is that, like XML, data files can be hierarchical and a "node" can have header and footer lines in the file.
>>
>> I wouldn't consider this a bug, just like XML DOM parsing is not a "bug". Of course, you're certainly entitled to your opinion.
>>
>> -David
>>
>>
>
>