Database Test Suite
Locked
 
|
I'm looking for a good way of testing service methods that do CRUD on
various tables in the database. I reviewed the framework/testtools and it didn't seem that there was much if anything going on there nor did it appear to address manipulation of test database data. While poking around I ran into ProposedOFBizTestFramework document by Brett Palmer that uses JXUnit (http://ofbizwiki.go-integral.com/Wiki.jsp?page=ProposedOFBizTestFramework ). This document looks quite promising and seems like it is exactly what I am looking for but the ofbiz-test that it referenced no longer exists at SourceForge. Is this a dead project? If not, where is it now located? What is the best practices for writing a test suite for new services that are manipulating data? Thanks! John |
|
|
I was a committer for DbUnit back in the day, which does stage
information nicely into the database for running tests. We used it in conjunction with WebTest to drive all testing from ant. It has worked well in the past, but might not be the most up to date solution. The only concern that I would have on this front is that there may be too much data to stage, With large amounts of data, the DbUnit approach could run into some performance issues with certain data sets. It might still be a viable approach for you John depending on your situation. I'd still love to hear more about Brett's JXUnit experience whenever he gets a chance. Cheers, Tim -- Tim Ruppert HotWax Media http://www.hotwaxmedia.com o:801.649.6594 f:801.649.6595 John Martin wrote: > I'm looking for a good way of testing service methods that do CRUD on > various tables in the database. > > I reviewed the framework/testtools and it didn't seem that there was > much if anything going on there nor did it appear to address > manipulation of test database data. > > While poking around I ran into ProposedOFBizTestFramework document by > Brett Palmer that uses JXUnit > (http://ofbizwiki.go-integral.com/Wiki.jsp?page=ProposedOFBizTestFramework > > ). > > This document looks quite promising and seems like it is exactly what > I am looking for but the ofbiz-test that it referenced no longer > exists at SourceForge. Is this a dead project? If not, where is it > now located? > > What is the best practices for writing a test suite for new services > that are manipulating data? > > Thanks! > > John |
|
|
Sorry, there has not been a lot of work done on the JXUnit stuff
since its original creation. The JXUnit project did not get a lot of support on sourceforge and the original contributors moved on to other projects. I continued to use the ofbiz jxunit code for some of my own projects but creating data driven tests were still difficult to create and maintain. Over the past year, I have been working with a different framework called grinder (http://grinder.sourceforge.net/). There have been a few posts on the subject from David Jones and Al Byers. Al has recently been working on it and we have shared some ideas on how to make it effective for ofbiz. Here are some of the features that I like about grinder: - Uses Jython as a the scripting language to implement tests. This makes it easier to create new tests without having to compile and deploy each time. - TCP Proxy listener to capture web requests and generates a test script. This script can then be modified to be more dynamic - Ability to simulate multiple clients with various load and frequency options Here are some things that are missing and what Al and I have been discussing: - Need to be able to separate data from the test - Need to develop some jython utilities that can read in a common XML (or something else) format for a test and pass the data to the jython test script Al and David can probably share some of their ideas on testing. We need to have more test tools but it is always a matter of priorities and everyone is always busy with other tasks. Thoughts? Brett On 10/23/06, Tim Ruppert <[hidden email]> wrote: > I was a committer for DbUnit back in the day, which does stage > information nicely into the database for running tests. We used it in > conjunction with WebTest to drive all testing from ant. It has worked > well in the past, but might not be the most up to date solution. > > The only concern that I would have on this front is that there may be > too much data to stage, With large amounts of data, the DbUnit approach > could run into some performance issues with certain data sets. It might > still be a viable approach for you John depending on your situation. > > I'd still love to hear more about Brett's JXUnit experience whenever he > gets a chance. > > Cheers, > Tim > > -- > Tim Ruppert > HotWax Media > http://www.hotwaxmedia.com > > o:801.649.6594 > f:801.649.6595 > > John Martin wrote: > > I'm looking for a good way of testing service methods that do CRUD on > > various tables in the database. > > > > I reviewed the framework/testtools and it didn't seem that there was > > much if anything going on there nor did it appear to address > > manipulation of test database data. > > > > While poking around I ran into ProposedOFBizTestFramework document by > > Brett Palmer that uses JXUnit > > (http://ofbizwiki.go-integral.com/Wiki.jsp?page=ProposedOFBizTestFramework > > > > ). > > > > This document looks quite promising and seems like it is exactly what > > I am looking for but the ofbiz-test that it referenced no longer > > exists at SourceForge. Is this a dead project? If not, where is it > > now located? > > > > What is the best practices for writing a test suite for new services > > that are manipulating data? > > > > Thanks! > > > > John > |
|
|
Brett, I've looked into Grinder earlier this year (albeit only for an
hour or so) and was pleasantly surprised by it. We used something called qftestJUI (http://www.qfs.de/en/qftestJUI/index.html) while developing a swing front end to a J2EE app - which had a lot of the record and playback stuff that we were looking for. Grinder seems to give you a lot of that, but some additional load testing stuff as well. Is that about right? Does anyone know how Grinder compares to Selinium and it's new Firefox plugin? Cheers, Tim -- Tim Ruppert HotWax Media http://www.hotwaxmedia.com o:801.649.6594 f:801.649.6595 Brett Palmer wrote: > Sorry, there has not been a lot of work done on the JXUnit stuff > since its original creation. The JXUnit project did not get a lot of > support on sourceforge and the original contributors moved on to other > projects. > > I continued to use the ofbiz jxunit code for some of my own projects > but creating data driven tests were still difficult to create and > maintain. > > Over the past year, I have been working with a different framework > called grinder (http://grinder.sourceforge.net/). There have been a > few posts on the subject from David Jones and Al Byers. Al has > recently been working on it and we have shared some ideas on how to > make it effective for ofbiz. > > Here are some of the features that I like about grinder: > > - Uses Jython as a the scripting language to implement tests. This > makes it easier to create new tests without having to compile and > deploy each time. > - TCP Proxy listener to capture web requests and generates a test > script. This script can then be modified to be more dynamic > - Ability to simulate multiple clients with various load and frequency > options > > > Here are some things that are missing and what Al and I have been > discussing: > > - Need to be able to separate data from the test > - Need to develop some jython utilities that can read in a common XML > (or something else) format for a test and pass the data to the jython > test script > > > Al and David can probably share some of their ideas on testing. We > need to have more test tools but it is always a matter of priorities > and everyone is always busy with other tasks. > > Thoughts? > > > Brett > > On 10/23/06, Tim Ruppert <[hidden email]> wrote: >> I was a committer for DbUnit back in the day, which does stage >> information nicely into the database for running tests. We used it in >> conjunction with WebTest to drive all testing from ant. It has worked >> well in the past, but might not be the most up to date solution. >> >> The only concern that I would have on this front is that there may be >> too much data to stage, With large amounts of data, the DbUnit approach >> could run into some performance issues with certain data sets. It might >> still be a viable approach for you John depending on your situation. >> >> I'd still love to hear more about Brett's JXUnit experience whenever he >> gets a chance. >> >> Cheers, >> Tim >> >> -- >> Tim Ruppert >> HotWax Media >> http://www.hotwaxmedia.com >> >> o:801.649.6594 >> f:801.649.6595 >> >> John Martin wrote: >> > I'm looking for a good way of testing service methods that do CRUD on >> > various tables in the database. >> > >> > I reviewed the framework/testtools and it didn't seem that there was >> > much if anything going on there nor did it appear to address >> > manipulation of test database data. >> > >> > While poking around I ran into ProposedOFBizTestFramework document by >> > Brett Palmer that uses JXUnit >> > >> (http://ofbizwiki.go-integral.com/Wiki.jsp?page=ProposedOFBizTestFramework >> >> > >> > ). >> > >> > This document looks quite promising and seems like it is exactly what >> > I am looking for but the ofbiz-test that it referenced no longer >> > exists at SourceForge. Is this a dead project? If not, where is it >> > now located? >> > >> > What is the best practices for writing a test suite for new services >> > that are manipulating data? >> > >> > Thanks! >> > >> > John >> |
|
|
Grinder does look really interesting and appears to me to be geared
more for testing the application for a UI and performance perspective. What I am really after is something that is more database oriented to allow a test suite to initialize data appropriately, run the test, and then clean out the test data. I'm writing new services to insert/update data and want to have a simple way of building tests to make sure that my code is working correctly. Currently, I write a test script that invokes the service and I inspect the results and walk through the code in the debugger. I agree with Brett that "creating data driven tests were still difficult to create and maintain." There's got to be an easier way... Brett - I would be interested in the JXUnit integration with ofBiz since that appears to be inline with what I'm trying to do. Can point me to where I can get what I would need to use it? Thanks, John |
|
|
John, this is exactly what I used DbUnit (http://www.dbunit.org/) for in
the past. You can drive this from Ant or thru code, so it's as flexible as you need it to be. Check out some of the information up on their site - as well as this (slightly dated) article written about them - http://www.onjava.com/pub/a/onjava/2004/01/21/dbunit.html . What we did was extend their DatabaseTestCase and load and clear data based upon the test case or test suite that we were running - using dynamically generated resource paths based upon the package and class names of the files. If you're interested and I can be of help - I'm here to answer more Q's Cheers, Tim -- Tim Ruppert HotWax Media http://www.hotwaxmedia.com o:801.649.6594 f:801.649.6595 John Martin wrote: > Grinder does look really interesting and appears to me to be geared > more for testing the application for a UI and performance perspective. > > What I am really after is something that is more database oriented to > allow a test suite to initialize data appropriately, run the test, and > then clean out the test data. I'm writing new services to > insert/update data and want to have a simple way of building tests to > make sure that my code is working correctly. > > Currently, I write a test script that invokes the service and I > inspect the results and walk through the code in the debugger. I > agree with Brett that "creating data driven tests were still difficult > to create and maintain." There's got to be an easier way... > > Brett - I would be interested in the JXUnit integration with ofBiz > since that appears to be inline with what I'm trying to do. Can point > me to where I can get what I would need to use it? > > Thanks, > > John |
|
|
In reply to this post by John Martin
John, Part of the test plan for OFBiz (that has been around for well over a year has not seen much attention) is to be able to easily to data level testing. We plan to use the OOTB demo and seed data as the initial database state, and then after tests are done the test database would simply be thrown away (avoiding the need to "undo" all test changes). To validate results on the data level there is an existing feature in the simple-method that we plan to make easier to use in the test config XML files. What it does is read an entity engine XML file (which can exported from WebTools, or created manually) and validate it against the database by asserting that each element in the XML file matches a database record. It reports records that are missing and all fields that are different for existing records. Is that more or less what you are looking for? The intent with these is to be intermixed with UI level tests and test services and such to provide testing facilities for the various architectural layers in OFBiz. -David On Oct 24, 2006, at 6:54 AM, John Martin wrote: > Grinder does look really interesting and appears to me to be geared > more for testing the application for a UI and performance perspective. > > What I am really after is something that is more database oriented to > allow a test suite to initialize data appropriately, run the test, and > then clean out the test data. I'm writing new services to > insert/update data and want to have a simple way of building tests to > make sure that my code is working correctly. > > Currently, I write a test script that invokes the service and I > inspect the results and walk through the code in the debugger. I > agree with Brett that "creating data driven tests were still difficult > to create and maintain." There's got to be an easier way... > > Brett - I would be interested in the JXUnit integration with ofBiz > since that appears to be inline with what I'm trying to do. Can point > me to where I can get what I would need to use it? > > Thanks, > > John |
|
|
Hi David,
I think that the "throw away" database idea is probably out of the question for my needs. I code in an iterative process and need to repeat the tests through the coding/debugging cycle. When the code is complete, so is the test suite for that service or set of services which is then available for regression testing in the system. During the development cycle I could run the tests 10-30 or more times depending on the complexity. If I understand your solution, one would need to repopulate the database after each iteration which would be very time consuming. In addition, I could see issues with the XML entity comparision with createXXX and lastUpdatedXXX datetime columns as well as the autoincrement id columns. What I am really looking for is the junit functionality where the test suite has a build up and tear down section that is called before/after the tests. If this build up / tear down could be driven with XML, that would be best. It seems that the dbUnit project is more in line with what I am looking for. I'm going to explore this option and will colaborate with Tim on this. Thanks, John On 10/24/06, David E Jones <[hidden email]> wrote: > > John, > > Part of the test plan for OFBiz (that has been around for well over a > year has not seen much attention) is to be able to easily to data > level testing. We plan to use the OOTB demo and seed data as the > initial database state, and then after tests are done the test > database would simply be thrown away (avoiding the need to "undo" all > test changes). > > To validate results on the data level there is an existing feature in > the simple-method that we plan to make easier to use in the test > config XML files. What it does is read an entity engine XML file > (which can exported from WebTools, or created manually) and validate > it against the database by asserting that each element in the XML > file matches a database record. It reports records that are missing > and all fields that are different for existing records. > > Is that more or less what you are looking for? The intent with these > is to be intermixed with UI level tests and test services and such to > provide testing facilities for the various architectural layers in > OFBiz. > > -David > > > On Oct 24, 2006, at 6:54 AM, John Martin wrote: > > > Grinder does look really interesting and appears to me to be geared > > more for testing the application for a UI and performance perspective. > > > > What I am really after is something that is more database oriented to > > allow a test suite to initialize data appropriately, run the test, and > > then clean out the test data. I'm writing new services to > > insert/update data and want to have a simple way of building tests to > > make sure that my code is working correctly. > > > > Currently, I write a test script that invokes the service and I > > inspect the results and walk through the code in the debugger. I > > agree with Brett that "creating data driven tests were still difficult > > to create and maintain." There's got to be an easier way... > > > > Brett - I would be interested in the JXUnit integration with ofBiz > > since that appears to be inline with what I'm trying to do. Can point > > me to where I can get what I would need to use it? > > > > Thanks, > > > > John > > |
|
|
John, Okay, that's a good distinction. You're not looking for regression or system wide tests, which generally take enough time to run that building a test database (about 3 minutes on Derby on my laptop) is not a significant amount of time. For quick tests like this I'd recommend creating test services that do the build up, code run, and results test all at once and then just roll back the transaction to do the cleanup. This tends to work fairly well and leverages existing transaction functionality that is pretty solid and does the "undo" easily. This is a lot easier than trying to save the state of the db and restoring it in the tear down. Of course, if you're not worried about developer efficiency, then code away and have a good time... ;) -David On Oct 24, 2006, at 6:07 PM, John Martin wrote: > Hi David, > > I think that the "throw away" database idea is probably out of the > question for my needs. I code in an iterative process and need to > repeat the tests through the coding/debugging cycle. When the code is > complete, so is the test suite for that service or set of services > which is then available for regression testing in the system. > > During the development cycle I could run the tests 10-30 or more times > depending on the complexity. If I understand your solution, one would > need to repopulate the database after each iteration which would be > very time consuming. In addition, I could see issues with the XML > entity comparision with createXXX and lastUpdatedXXX datetime columns > as well as the autoincrement id columns. > > What I am really looking for is the junit functionality where the test > suite has a build up and tear down section that is called before/after > the tests. If this build up / tear down could be driven with XML, > that would be best. > > It seems that the dbUnit project is more in line with what I am > looking for. I'm going to explore this option and will colaborate > with Tim on this. > > Thanks, > > John > > On 10/24/06, David E Jones <[hidden email]> wrote: >> >> John, >> >> Part of the test plan for OFBiz (that has been around for well over a >> year has not seen much attention) is to be able to easily to data >> level testing. We plan to use the OOTB demo and seed data as the >> initial database state, and then after tests are done the test >> database would simply be thrown away (avoiding the need to "undo" all >> test changes). >> >> To validate results on the data level there is an existing feature in >> the simple-method that we plan to make easier to use in the test >> config XML files. What it does is read an entity engine XML file >> (which can exported from WebTools, or created manually) and validate >> it against the database by asserting that each element in the XML >> file matches a database record. It reports records that are missing >> and all fields that are different for existing records. >> >> Is that more or less what you are looking for? The intent with these >> is to be intermixed with UI level tests and test services and such to >> provide testing facilities for the various architectural layers in >> OFBiz. >> >> -David >> >> >> On Oct 24, 2006, at 6:54 AM, John Martin wrote: >> >> > Grinder does look really interesting and appears to me to be geared >> > more for testing the application for a UI and performance >> perspective. >> > >> > What I am really after is something that is more database >> oriented to >> > allow a test suite to initialize data appropriately, run the >> test, and >> > then clean out the test data. I'm writing new services to >> > insert/update data and want to have a simple way of building >> tests to >> > make sure that my code is working correctly. >> > >> > Currently, I write a test script that invokes the service and I >> > inspect the results and walk through the code in the debugger. I >> > agree with Brett that "creating data driven tests were still >> difficult >> > to create and maintain." There's got to be an easier way... >> > >> > Brett - I would be interested in the JXUnit integration with ofBiz >> > since that appears to be inline with what I'm trying to do. Can >> point >> > me to where I can get what I would need to use it? >> > >> > Thanks, >> > >> > John >> >> |
|
|
David,
Setting up the service and using transactions sound like a good idea. The only negative that I see initially is that some times inspecting the data visually is quite helpful to see what is going on. In that case, you have to let the transaction commit and then manually clean up the mess. Perhaps we could use the XML data import from with the build up method and the tear-down could intellegently remove data from the XML. Any thoughts on that? John |
|
|
On Oct 24, 2006, at 6:42 PM, John Martin wrote: > Setting up the service and using transactions sound like a good idea. > The only negative that I see initially is that some times inspecting > the data visually is quite helpful to see what is going on. In that > case, you have to let the transaction commit and then manually clean > up the mess. > > Perhaps we could use the XML data import from with the build up method > and the tear-down could intellegently remove data from the XML. Any > thoughts on that? Just one: depending on the approach you take it's either really hard to do, or requires a very large overhead and only somewhat hard to do (which might be okay for test environments). Either way, it requires defining exactly what you want, including the granularity of changes you want to deal with. The tough part is the "intelligent" part... what would that mean? -David |
|
|
Intelligent meaning that the teardown logic could refer to the test
data XML and perform the cleanup. The XML probably would need some syntax for the clean up. It would be a nice feature if you could disable the teardown so that you could visually inspect the data and then invoke the teardown on demand to do the cleanup. |
«
Return to OFBiz - User
|
1 view|%1 views
| Free forum by Nabble | Edit this page |