http://ofbiz.116.s1.nabble.com/framework-datafile-caching-tp1594248p1594390.html
> I'm sorry, was there a point in there somewhere?
>
> -David
>
>
> On Mar 15, 2010, at 9:00 PM, BJ Freeman wrote:
>
>> that reminds me of the wolf and the three pigs.
>> if you build something that has unrealistic requirements, then it will
>> not function in the real world.
>> the person making the requirements should be aware of what they are and
>> the limits or usage.
>> but to build something just to say you built is not what I call good design.
>>
>> Just to note, while I was documenting my way of doing things It was
>> suggested I look at the datafile way.
>>
>> I have and have attempted to use it in good faith.
>> I have realworld files that I handled long before coming to ofbiz that
>> are gigs in size, so the requirements have been around since the early 90's
>>
>>
>> ======================
>>
>> BJ Freeman
>>
http://bjfreeman.elance.com>> Strategic Power Office with Supplier Automation <
http://www.businessesnetwork.com/automation/viewforum.php?f=93>
>> Specialtymarket.com <
http://www.specialtymarket.com/>
>>
>> Systems Integrator-- Glad to Assist
>>
>> Chat Y! messenger: bjfr33man
>> Linkedin
>> <
http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>
>>
>>
>> David E Jones sent the following on 3/15/2010 6:48 PM:
>>> Sorry, that's silly. A bug can't exist without a requirement. Where was this requirement ever established? What in the design implies that this was a requirement? The API very clearly represents a process that reads the entire file into memory.
>>>
>>> It sounds like it doesn't meet a requirement that you came up with. That isn't the definition of a "bug", well except for the purposes of trolling on mailing lists I suppose.
>>>
>>> -David
>>>
>>>
>>> On Mar 15, 2010, at 7:16 PM, BJ Freeman wrote:
>>>
>>>> when code will not handle real world data, it is broken.
>>>> you and I discussed this when importing xml file and I choose to parse
>>>> them manually instead of using DOM.
>>>> So what term is there besides bug for broken design.
>>>>
>>>> =========================
>>>> BJ Freeman
>>>>
http://bjfreeman.elance.com>>>> Strategic Power Office with Supplier Automation <
http://www.businessesnetwork.com/automation/viewforum.php?f=93>
>>>> Specialtymarket.com <
http://www.specialtymarket.com/>
>>>>
>>>> Systems Integrator-- Glad to Assist
>>>>
>>>> Chat Y! messenger: bjfr33man
>>>> Linkedin
>>>> <
http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>
>>>>
>>>>
>>>> David E Jones sent the following on 3/15/2010 6:05 PM:
>>>>> On Mar 15, 2010, at 6:58 PM, Adam Heath wrote:
>>>>>
>>>>>> BJ Freeman wrote:
>>>>>>> one of my smaller import files (8mb) is taking forever to be read in and
>>>>>>> there is no output.
>>>>>>> I am seeing the memory rail against the max setting.
>>>>>>> got this error.
>>>>>> The datafile set of classes is very broken for large files. It has a
>>>>>> List<Record>, which means it will copy the entire file into memory
>>>>>> before doing anything with it. The datafile code is not designed to
>>>>>> handle large files.
>>>>>>
>>>>>> I consider this a bug that needs to be fixed. Checking...
>>>>> This should be possible, but may require API changes. The problem is that, like XML, data files can be hierarchical and a "node" can have header and footer lines in the file.
>>>>>
>>>>> I wouldn't consider this a bug, just like XML DOM parsing is not a "bug". Of course, you're certainly entitled to your opinion.
>>>>>
>>>>> -David
>>>>>
>>>>>
>>>
>>
>
>