Webtools generated XML directory doesn't import back in

[Toddnick]

Hello All,
I hit a snag on importing the XML from an ofbiz install. I used the webtools
to both generate the XML and attempt the import (import was called from
webtools/control/EntityImportDir). On the destination machine I uploaded the
exported files to a directory called xmlfiles. I used this as the path for
the import.

This is what the screen said:

Results:

Failed /home/user/xmldata/ReturnReason.xml adding to retry list for next
pass
<snip- lots of files listed here>

Failed /home/user/xmldata/ProdCatalogCategoryType.xml adding to retry list
for next pass

Pass 2 complete

---------------------------------------

Succeeded: 7 of 367

Failed: 360 of 367

---------------------------------------

Failed Files:

/home/user/xmldata/ReturnReason.xml
<snip- lots of files listed here>


Any ideas? I tried markmail and the docs website. I appreciate any help
Many thanks
--
Todd Burdeinei

[Jacques Le Roux]

This could be a key issue. One way to cope with that is to use "Check/Update Database" Webtools option.
Watch also for timeout in the screen and log

Jacques

From: "Todd Burdeinei" <[hidden email]>

> Hello All,
> I hit a snag on importing the XML from an ofbiz install. I used the webtools
> to both generate the XML and attempt the import (import was called from
> webtools/control/EntityImportDir). On the destination machine I uploaded the
> exported files to a directory called xmlfiles. I used this as the path for
> the import.
>
> This is what the screen said:
>
> Results:
>
> Failed /home/user/xmldata/ReturnReason.xml adding to retry list for next
> pass
> <snip- lots of files listed here>
>
> Failed /home/user/xmldata/ProdCatalogCategoryType.xml adding to retry list
> for next pass
>
> Pass 2 complete
>
> ---------------------------------------
>
> Succeeded: 7 of 367
>
> Failed: 360 of 367
>
> ---------------------------------------
>
> Failed Files:
>
> /home/user/xmldata/ReturnReason.xml
> <snip- lots of files listed here>
>
>
> Any ideas? I tried markmail and the docs website. I appreciate any help
> Many thanks
> --
> Todd Burdeinei
>

[BJ Freeman]

the problem, from my experience, is that there are relationships PFK
that do not get resolved.
the only way I have found to import is to make these all part of the
intial import through the build file, ofbiz-component.xml using the seed
reader, so you explicitly state which files get imported in what sequence.
then do an
ant run-install-seed

Todd Burdeinei sent the following on 10/16/2008 2:40 AM:

> Hello All,
> I hit a snag on importing the XML from an ofbiz install. I used the webtools
> to both generate the XML and attempt the import (import was called from
> webtools/control/EntityImportDir). On the destination machine I uploaded the
> exported files to a directory called xmlfiles. I used this as the path for
> the import.
>
> This is what the screen said:
>
> Results:
>
> Failed /home/user/xmldata/ReturnReason.xml adding to retry list for next
> pass
> <snip- lots of files listed here>
>
> Failed /home/user/xmldata/ProdCatalogCategoryType.xml adding to retry list
> for next pass
>
> Pass 2 complete
>
> ---------------------------------------
>
> Succeeded: 7 of 367
>
> Failed: 360 of 367
>
> ---------------------------------------
>
> Failed Files:
>
> /home/user/xmldata/ReturnReason.xml
> <snip- lots of files listed here>
>
>
> Any ideas? I tried markmail and the docs website. I appreciate any help
> Many thanks

[David E Jones]

There are various ways to resolve the issue with foreign keys, and  
sorting files is not the easiest or most effective way to do this.  
BTW, the import directory service can kind of figure out the file  
order for you by trying a directory of files over and over and in each  
pass removing the successful files until all are loaded (or hopefully  
all are loaded). There is no need to do it manually (and on a side  
note, there are easier ways to do it manually than setting up your  
files as seed data).

A better approach is to just select the "dummy fks" checkbox and it  
will resolve missing foreign keys by adding a dummy record with just  
the pk. Use this only if you're pretty sure that all foreign keys will  
eventually be satisfied, otherwise you end up with "hanging" records  
that have no data and get around what foreign keys are meant to tell  
you, ie that data is missing that your data depends on. Still, the  
"dummy fks" approach is the only way to get around circular  
dependencies.

Another option, that Jacques kind of mentioned, is to use the Check/
Update Database page to remove all fks and then load your data, then  
use that page again to add all fks back (And it will tell you if any  
fks are violated when they are added back).

-David


On Oct 16, 2008, at 10:50 AM, BJ Freeman wrote:

> the problem, from my experience, is that there are relationships PFK
> that do not get resolved.
> the only way I have found to import is to make these all part of the
> intial import through the build file, ofbiz-component.xml using the  
> seed
> reader, so you explicitly state which files get imported in what  
> sequence.
> then do an
> ant run-install-seed
>
> Todd Burdeinei sent the following on 10/16/2008 2:40 AM:
>> Hello All,
>> I hit a snag on importing the XML from an ofbiz install. I used the  
>> webtools
>> to both generate the XML and attempt the import (import was called  
>> from
>> webtools/control/EntityImportDir). On the destination machine I  
>> uploaded the
>> exported files to a directory called xmlfiles. I used this as the  
>> path for
>> the import.
>>
>> This is what the screen said:
>>
>> Results:
>>
>> Failed /home/user/xmldata/ReturnReason.xml adding to retry list for  
>> next
>> pass
>> <snip- lots of files listed here>
>>
>> Failed /home/user/xmldata/ProdCatalogCategoryType.xml adding to  
>> retry list
>> for next pass
>>
>> Pass 2 complete
>>
>> ---------------------------------------
>>
>> Succeeded: 7 of 367
>>
>> Failed: 360 of 367
>>
>> ---------------------------------------
>>
>> Failed Files:
>>
>> /home/user/xmldata/ReturnReason.xml
>> <snip- lots of files listed here>
>>
>>
>> Any ideas? I tried markmail and the docs website. I appreciate any  
>> help
>> Many thanks

[BJ Freeman]

I tried those David. never could get a 100% and like you mention then
you have the efforts to figure out what went wrong.


David E Jones sent the following on 10/16/2008 6:07 PM:

>
> There are various ways to resolve the issue with foreign keys, and
> sorting files is not the easiest or most effective way to do this. BTW,
> the import directory service can kind of figure out the file order for
> you by trying a directory of files over and over and in each pass
> removing the successful files until all are loaded (or hopefully all are
> loaded). There is no need to do it manually (and on a side note, there
> are easier ways to do it manually than setting up your files as seed data).
>
> A better approach is to just select the "dummy fks" checkbox and it will
> resolve missing foreign keys by adding a dummy record with just the pk.
> Use this only if you're pretty sure that all foreign keys will
> eventually be satisfied, otherwise you end up with "hanging" records
> that have no data and get around what foreign keys are meant to tell
> you, ie that data is missing that your data depends on. Still, the
> "dummy fks" approach is the only way to get around circular dependencies.
>
> Another option, that Jacques kind of mentioned, is to use the
> Check/Update Database page to remove all fks and then load your data,
> then use that page again to add all fks back (And it will tell you if
> any fks are violated when they are added back).
>
> -David
>
>
> On Oct 16, 2008, at 10:50 AM, BJ Freeman wrote:
>
>> the problem, from my experience, is that there are relationships PFK
>> that do not get resolved.
>> the only way I have found to import is to make these all part of the
>> intial import through the build file, ofbiz-component.xml using the seed
>> reader, so you explicitly state which files get imported in what
>> sequence.
>> then do an
>> ant run-install-seed
>>
>> Todd Burdeinei sent the following on 10/16/2008 2:40 AM:
>>> Hello All,
>>> I hit a snag on importing the XML from an ofbiz install. I used the
>>> webtools
>>> to both generate the XML and attempt the import (import was called from
>>> webtools/control/EntityImportDir). On the destination machine I
>>> uploaded the
>>> exported files to a directory called xmlfiles. I used this as the
>>> path for
>>> the import.
>>>
>>> This is what the screen said:
>>>
>>> Results:
>>>
>>> Failed /home/user/xmldata/ReturnReason.xml adding to retry list for next
>>> pass
>>> <snip- lots of files listed here>
>>>
>>> Failed /home/user/xmldata/ProdCatalogCategoryType.xml adding to retry
>>> list
>>> for next pass
>>>
>>> Pass 2 complete
>>>
>>> ---------------------------------------
>>>
>>> Succeeded: 7 of 367
>>>
>>> Failed: 360 of 367
>>>
>>> ---------------------------------------
>>>
>>> Failed Files:
>>>
>>> /home/user/xmldata/ReturnReason.xml
>>> <snip- lots of files listed here>
>>>
>>>
>>> Any ideas? I tried markmail and the docs website. I appreciate any help
>>> Many thanks
>
>
>