All JUnit tests now pass! Next steps...
Locked
12
12
All JUnit tests now pass! Next steps...
 
|
It took a while but all our JUnit tests now pass, next I'd like to get
a continuous integration server set up. Can anybody recommend some tools that they've had good experiences with? I'm trying out Apache Continuum at the moment (eat our own dog food) but I'd love to hear from anyone who has actual experience with an open source CI tool (I do not). Once that is done I'd like to have it run the tests after every commit and report any failures to the dev list. It would then be the offending committer's responsibility (well primarily at least) to fix the problem as soon as possible, much like a build failure. If we can't get everyone to agree to take on that responsibility then I might as well stop now because I'll be damned if I'm going to spend any more time fixing tests that I didn't break :-) Any thoughts? After that I'll starting looking at what can be done to improve the test tools and make tests easier to write, start looking at selenium and start writing tests for open bug reports to be committed with the fix so that they never occur again. All of this should help us increase the stability of the trunk and our confidence when taking a checkout that we won't half to spend our development time fixing things that used to work. It'll hopefully also encourage the community to contribute more tests with the knowledge that doing so will increase the stability of the functionality they depend on. Fix a bug and you're good for a day, write a test and you're good a lifetime :-) Regards Scott |
smime.p7s (4K)
Re: All JUnit tests now pass! Next steps...
|
|
Re: All JUnit tests now pass! Next steps...
|
Re: All JUnit tests now pass! Next steps...
Administrator
|
Re: All JUnit tests now pass! Next steps...
|
|
Re: All JUnit tests now pass! Next steps...
|
|
In reply to this post by Christian Geisert
Thanks for the pointer Christian, I figured the ASF would have
something for us and they haven't disappointed :-) I'll keep playing around the different CI services locally for a little bit so I can learn how they work a bit better and also find the one that will be the most useful to us. Thanks Scott On 20/11/2009, at 1:08 AM, Christian Geisert wrote: > Scott Gray schrieb: >> It took a while but all our JUnit tests now pass, next I'd like to >> get a continuous integration server set up. Can anybody recommend >> some tools > > Why not just use what is available at the ASF ;-) > > http://ci.apache.org/ > > I use Hudson and like it, but I haven't tried the others... > >> Once that is done I'd like to have it run the tests after every >> commit and report any failures to the dev list. It would then be >> the offending committer's responsibility (well primarily at least) >> to fix the problem as soon as possible, much like a build failure. >> If we can't get everyone to agree to take on that responsibility >> then I might as well stop now because I'll be damned if I'm going >> to spend any more time fixing tests that I didn't break :-) > > +1 > >> All of this should help us increase the stability of the trunk and >> our confidence when taking a checkout that we won't half to spend >> our development time fixing things that used to work. It'll >> hopefully also encourage the community to contribute more tests >> with the knowledge that doing so will increase the stability of the >> functionality they depend on. Fix a bug and you're good for a day, >> write a test and you're good a lifetime :-) > > Absolutly agreed, big thanks for your (and all the others) work in > this area! > > -- > Christian > |
Re: All JUnit tests now pass! Next steps...
|
|
In reply to this post by Adrian Crum
On 20/11/2009, at 4:43 AM, Adrian Crum wrote:
> I would like to build out the unit tests in the framework/base > component, but I still need help getting that component's build.xml > file to run tests. Sorry I meant to look into it when you first raised that thread, I'll dig it up and see if I can help. Regards Scott |
Re: All JUnit tests now pass! Next steps...
|
|
Re: All JUnit tests now pass! Next steps...
|
|
Re: All JUnit tests now pass! Next steps...
|
|
In reply to this post by Scott Gray-2
I've used Cruise Control and Bamboo in the past and they're both
amazing. I'm sure that CI will also do the trick. Whoever's spending the time on the new Virtual Boxes that have been setup for OFBiz should totally get this setup as well. It would be a big win. Cheers, Ruppert -- Tim Ruppert HotWax Media http://www.hotwaxmedia.com o:801.649.6594 f:801.649.6595 On Nov 19, 2009, at 1:06 PM, Scott Gray wrote: > Thanks for the pointer Christian, I figured the ASF would have > something for us and they haven't disappointed :-) > > I'll keep playing around the different CI services locally for a > little bit so I can learn how they work a bit better and also find > the one that will be the most useful to us. > > Thanks > Scott > > On 20/11/2009, at 1:08 AM, Christian Geisert wrote: > >> Scott Gray schrieb: >>> It took a while but all our JUnit tests now pass, next I'd like to >>> get a continuous integration server set up. Can anybody recommend >>> some tools >> >> Why not just use what is available at the ASF ;-) >> >> http://ci.apache.org/ >> >> I use Hudson and like it, but I haven't tried the others... >> >>> Once that is done I'd like to have it run the tests after every >>> commit and report any failures to the dev list. It would then be >>> the offending committer's responsibility (well primarily at least) >>> to fix the problem as soon as possible, much like a build >>> failure. If we can't get everyone to agree to take on that >>> responsibility then I might as well stop now because I'll be >>> damned if I'm going to spend any more time fixing tests that I >>> didn't break :-) >> >> +1 >> >>> All of this should help us increase the stability of the trunk and >>> our confidence when taking a checkout that we won't half to spend >>> our development time fixing things that used to work. It'll >>> hopefully also encourage the community to contribute more tests >>> with the knowledge that doing so will increase the stability of >>> the functionality they depend on. Fix a bug and you're good for a >>> day, write a test and you're good a lifetime :-) >> >> Absolutly agreed, big thanks for your (and all the others) work in >> this area! >> >> -- >> Christian >> > |
Re: All JUnit tests now pass! Next steps...
|
|
In reply to this post by Adrian Crum
You can run a single test suite using the main build file:
ant run-single-test-suite -Dtest.component=accounting - Dtest.suiteName=accountingtests it doesn't depend on the build target for some reason though but that could easily be changed. Is this what you meant or something different? Regards Scott On 20/11/2009, at 9:20 AM, Adrian Crum wrote: > What I had in mind was something like a run-tests ant target in the > component's build.xml file. The run-tests target would depend on the > build target. > > That would make code testing really easy and convenient: modify > code, run-tests, modify code some more, run-tests... > > -Adrian > > Scott Gray wrote: >> On 20/11/2009, at 4:43 AM, Adrian Crum wrote: >>> I would like to build out the unit tests in the framework/base >>> component, but I still need help getting that component's >>> build.xml file to run tests. >> Sorry I meant to look into it when you first raised that thread, >> I'll dig it up and see if I can help. >> Regards >> Scott |
Re: All JUnit tests now pass! Next steps...
|
|
In reply to this post by Tim Ruppert
CI just stands for continuous integration and is the umbrella that all
the specific tools live under. I haven't (and don't plan to at this stage) had a look at the ASF hosted OFBiz demo setup but we should be able to take care of this as a separate effort because the CI tools are hosted on their own separate servers. From the link that Christian provided, we've got the choice of: BuildBot (Feature list looks impressive: http://ci.apache.org/buildbot.html , includes the RAT tool for licensing checks) Continuum (ASF project) Gump (ASF project) Hudson (Seems to be popular) I'd like to prefer an ASF project but not if it means giving up on some features that would be really useful. Regards Scott On 20/11/2009, at 9:26 AM, Tim Ruppert wrote: > I've used Cruise Control and Bamboo in the past and they're both > amazing. I'm sure that CI will also do the trick. > > Whoever's spending the time on the new Virtual Boxes that have been > setup for OFBiz should totally get this setup as well. It would be > a big win. > > Cheers, > Ruppert > -- > Tim Ruppert > HotWax Media > http://www.hotwaxmedia.com > > o:801.649.6594 > f:801.649.6595 > > On Nov 19, 2009, at 1:06 PM, Scott Gray wrote: > >> Thanks for the pointer Christian, I figured the ASF would have >> something for us and they haven't disappointed :-) >> >> I'll keep playing around the different CI services locally for a >> little bit so I can learn how they work a bit better and also find >> the one that will be the most useful to us. >> >> Thanks >> Scott >> >> On 20/11/2009, at 1:08 AM, Christian Geisert wrote: >> >>> Scott Gray schrieb: >>>> It took a while but all our JUnit tests now pass, next I'd like >>>> to get a continuous integration server set up. Can anybody >>>> recommend some tools >>> >>> Why not just use what is available at the ASF ;-) >>> >>> http://ci.apache.org/ >>> >>> I use Hudson and like it, but I haven't tried the others... >>> >>>> Once that is done I'd like to have it run the tests after every >>>> commit and report any failures to the dev list. It would then be >>>> the offending committer's responsibility (well primarily at >>>> least) to fix the problem as soon as possible, much like a build >>>> failure. If we can't get everyone to agree to take on that >>>> responsibility then I might as well stop now because I'll be >>>> damned if I'm going to spend any more time fixing tests that I >>>> didn't break :-) >>> >>> +1 >>> >>>> All of this should help us increase the stability of the trunk >>>> and our confidence when taking a checkout that we won't half to >>>> spend our development time fixing things that used to work. >>>> It'll hopefully also encourage the community to contribute more >>>> tests with the knowledge that doing so will increase the >>>> stability of the functionality they depend on. Fix a bug and >>>> you're good for a day, write a test and you're good a lifetime :-) >>> >>> Absolutly agreed, big thanks for your (and all the others) work in >>> this area! >>> >>> -- >>> Christian >>> >> > |
Re: All JUnit tests now pass! Next steps...
|
|
Re: All JUnit tests now pass! Next steps...
|
|
Just because our test tools require container loading there is no
reason why we couldn't run tests that don't require containers and use the junit ant task to run them during the test build: http://junit.sourceforge.net/doc/faq/faq.htm#running_5 http://ant.apache.org/manual/OptionalTasks/junit.html That reminds me, we need to upgrade to the latest version of JUnit, I'll take care of it this weekend. Regards Scott On 20/11/2009, at 10:00 AM, Adrian Crum wrote: > I guess I was being myopic. Adam mentioned having to load the > containers, and he's right - most component-level tests would > require loading OFBiz in order for them to work. I was thinking only > of the framework/base component - where you could run unit tests on > the various classes found there. That wouldn't require container > loading (I think). > > Maybe it's too much of a bother. *shrug* > > -Adrian > > > Scott Gray wrote: >> You can run a single test suite using the main build file: >> ant run-single-test-suite -Dtest.component=accounting - >> Dtest.suiteName=accountingtests >> it doesn't depend on the build target for some reason though but >> that could easily be changed. >> Is this what you meant or something different? >> Regards >> Scott >> On 20/11/2009, at 9:20 AM, Adrian Crum wrote: >>> What I had in mind was something like a run-tests ant target in >>> the component's build.xml file. The run-tests target would depend >>> on the build target. >>> >>> That would make code testing really easy and convenient: modify >>> code, run-tests, modify code some more, run-tests... >>> >>> -Adrian >>> >>> Scott Gray wrote: >>>> On 20/11/2009, at 4:43 AM, Adrian Crum wrote: >>>>> I would like to build out the unit tests in the framework/base >>>>> component, but I still need help getting that component's >>>>> build.xml file to run tests. >>>> Sorry I meant to look into it when you first raised that thread, >>>> I'll dig it up and see if I can help. >>>> Regards >>>> Scott |
Re: All JUnit tests now pass! Next steps...
|
|
Re: All JUnit tests now pass! Next steps...
|
|
In reply to this post by Scott Gray-2
Quick update, I've since learnt that we've already had the trunk and
9.04 branch running on buildbot at the ASF for a few months now. I've created an infra ticket (https://issues.apache.org/jira/browse/INFRA-2345 ) requesting that build notifications be sent to the dev list and also that the run-install and run-tests targets be added to the buildbot configuration (for the trunk only). Regards Scott On 20/11/2009, at 9:06 AM, Scott Gray wrote: > Thanks for the pointer Christian, I figured the ASF would have > something for us and they haven't disappointed :-) > > I'll keep playing around the different CI services locally for a > little bit so I can learn how they work a bit better and also find > the one that will be the most useful to us. > > Thanks > Scott > > On 20/11/2009, at 1:08 AM, Christian Geisert wrote: > >> Scott Gray schrieb: >>> It took a while but all our JUnit tests now pass, next I'd like to >>> get a continuous integration server set up. Can anybody recommend >>> some tools >> >> Why not just use what is available at the ASF ;-) >> >> http://ci.apache.org/ >> >> I use Hudson and like it, but I haven't tried the others... >> >>> Once that is done I'd like to have it run the tests after every >>> commit and report any failures to the dev list. It would then be >>> the offending committer's responsibility (well primarily at least) >>> to fix the problem as soon as possible, much like a build >>> failure. If we can't get everyone to agree to take on that >>> responsibility then I might as well stop now because I'll be >>> damned if I'm going to spend any more time fixing tests that I >>> didn't break :-) >> >> +1 >> >>> All of this should help us increase the stability of the trunk and >>> our confidence when taking a checkout that we won't half to spend >>> our development time fixing things that used to work. It'll >>> hopefully also encourage the community to contribute more tests >>> with the knowledge that doing so will increase the stability of >>> the functionality they depend on. Fix a bug and you're good for a >>> day, write a test and you're good a lifetime :-) >> >> Absolutly agreed, big thanks for your (and all the others) work in >> this area! >> >> -- >> Christian >> > |
Re: All JUnit tests now pass! Next steps...
|
|
Awesome Scott - thanks.
Cheers, Ruppert -- Tim Ruppert HotWax Media http://www.hotwaxmedia.com o:801.649.6594 f:801.649.6595 On Nov 23, 2009, at 8:53 PM, Scott Gray wrote: > Quick update, I've since learnt that we've already had the trunk and > 9.04 branch running on buildbot at the ASF for a few months now. > I've created an infra ticket (https://issues.apache.org/jira/browse/INFRA-2345 > ) requesting that build notifications be sent to the dev list and > also that the run-install and run-tests targets be added to the > buildbot configuration (for the trunk only). > > Regards > Scott > > On 20/11/2009, at 9:06 AM, Scott Gray wrote: > >> Thanks for the pointer Christian, I figured the ASF would have >> something for us and they haven't disappointed :-) >> >> I'll keep playing around the different CI services locally for a >> little bit so I can learn how they work a bit better and also find >> the one that will be the most useful to us. >> >> Thanks >> Scott >> >> On 20/11/2009, at 1:08 AM, Christian Geisert wrote: >> >>> Scott Gray schrieb: >>>> It took a while but all our JUnit tests now pass, next I'd like >>>> to get a continuous integration server set up. Can anybody >>>> recommend some tools >>> >>> Why not just use what is available at the ASF ;-) >>> >>> http://ci.apache.org/ >>> >>> I use Hudson and like it, but I haven't tried the others... >>> >>>> Once that is done I'd like to have it run the tests after every >>>> commit and report any failures to the dev list. It would then be >>>> the offending committer's responsibility (well primarily at >>>> least) to fix the problem as soon as possible, much like a build >>>> failure. If we can't get everyone to agree to take on that >>>> responsibility then I might as well stop now because I'll be >>>> damned if I'm going to spend any more time fixing tests that I >>>> didn't break :-) >>> >>> +1 >>> >>>> All of this should help us increase the stability of the trunk >>>> and our confidence when taking a checkout that we won't half to >>>> spend our development time fixing things that used to work. >>>> It'll hopefully also encourage the community to contribute more >>>> tests with the knowledge that doing so will increase the >>>> stability of the functionality they depend on. Fix a bug and >>>> you're good for a day, write a test and you're good a lifetime :-) >>> >>> Absolutly agreed, big thanks for your (and all the others) work in >>> this area! >>> >>> -- >>> Christian >>> >> > |
Re: All JUnit tests now pass! Next steps...
|
|
Re: All JUnit tests now pass! Next steps...
|
|
Hi Adrian,
Yeah that issue has been there for a while but you'll notice that the exception is just a logged exception and isn't actually thrown, hence no error and the test passes. Regards Scott On 24/11/2009, at 6:01 PM, Adrian Crum wrote: > Scott, > > Are you sure all tests pass? I'm seeing errors in the entity engine > test - testBlobCreate. I thought it might be caused by my recent > Blob converter commit, so I reverted it locally. I still get the > same error. Looking at the code causing the error > (GenericEntity.java line 420) it appears to me this test never > should have succeeded. > > -Adrian > > --- On Mon, 11/23/09, Scott Gray <[hidden email]> wrote: > >> From: Scott Gray <[hidden email]> >> Subject: Re: All JUnit tests now pass! Next steps... >> To: [hidden email] >> Date: Monday, November 23, 2009, 7:53 PM >> Quick update, I've since learnt that >> we've already had the trunk and 9.04 branch running on >> buildbot at the ASF for a few months now. I've created >> an infra ticket (https://issues.apache.org/jira/browse/INFRA-2345) >> requesting that build notifications be sent to the dev list >> and also that the run-install and run-tests targets be added >> to the buildbot configuration (for the trunk only). >> >> Regards >> Scott >> >> On 20/11/2009, at 9:06 AM, Scott Gray wrote: >> >>> Thanks for the pointer Christian, I figured the ASF >> would have something for us and they haven't disappointed >> :-) >>> >>> I'll keep playing around the different CI services >> locally for a little bit so I can learn how they work a bit >> better and also find the one that will be the most useful to >> us. >>> >>> Thanks >>> Scott >>> >>> On 20/11/2009, at 1:08 AM, Christian Geisert wrote: >>> >>>> Scott Gray schrieb: >>>>> It took a while but all our JUnit tests now >> pass, next I'd like to get a continuous integration server >> set up. Can anybody recommend some tools >>>> >>>> Why not just use what is available at the ASF ;-) >>>> >>>> http://ci.apache.org/ >>>> >>>> I use Hudson and like it, but I haven't tried the >> others... >>>> >>>>> Once that is done I'd like to have it run the >> tests after every commit and report any failures to the dev >> list. It would then be the offending committer's >> responsibility (well primarily at least) to fix the problem >> as soon as possible, much like a build failure. If we >> can't get everyone to agree to take on that responsibility >> then I might as well stop now because I'll be damned if I'm >> going to spend any more time fixing tests that I didn't >> break :-) >>>> >>>> +1 >>>> >>>>> All of this should help us increase the >> stability of the trunk and our confidence when taking a >> checkout that we won't half to spend our development time >> fixing things that used to work. It'll hopefully also >> encourage the community to contribute more tests with the >> knowledge that doing so will increase the stability of the >> functionality they depend on. Fix a bug and you're >> good for a day, write a test and you're good a lifetime :-) >>>> >>>> Absolutly agreed, big thanks for your (and all the >> others) work in this area! >>>> >>>> --Christian >>>> >>> >> >> > > > |
Re: All JUnit tests now pass! Next steps...
|
|
BTW, the accounting tests are failing at the moment but I'll have them
fixed again shortly. Regards Scott On 24/11/2009, at 6:09 PM, Scott Gray wrote: > Hi Adrian, > > Yeah that issue has been there for a while but you'll notice that > the exception is just a logged exception and isn't actually thrown, > hence no error and the test passes. > > Regards > Scott > > On 24/11/2009, at 6:01 PM, Adrian Crum wrote: > >> Scott, >> >> Are you sure all tests pass? I'm seeing errors in the entity engine >> test - testBlobCreate. I thought it might be caused by my recent >> Blob converter commit, so I reverted it locally. I still get the >> same error. Looking at the code causing the error >> (GenericEntity.java line 420) it appears to me this test never >> should have succeeded. >> >> -Adrian >> >> --- On Mon, 11/23/09, Scott Gray <[hidden email]> wrote: >> >>> From: Scott Gray <[hidden email]> >>> Subject: Re: All JUnit tests now pass! Next steps... >>> To: [hidden email] >>> Date: Monday, November 23, 2009, 7:53 PM >>> Quick update, I've since learnt that >>> we've already had the trunk and 9.04 branch running on >>> buildbot at the ASF for a few months now. I've created >>> an infra ticket (https://issues.apache.org/jira/browse/INFRA-2345) >>> requesting that build notifications be sent to the dev list >>> and also that the run-install and run-tests targets be added >>> to the buildbot configuration (for the trunk only). >>> >>> Regards >>> Scott >>> >>> On 20/11/2009, at 9:06 AM, Scott Gray wrote: >>> >>>> Thanks for the pointer Christian, I figured the ASF >>> would have something for us and they haven't disappointed >>> :-) >>>> >>>> I'll keep playing around the different CI services >>> locally for a little bit so I can learn how they work a bit >>> better and also find the one that will be the most useful to >>> us. >>>> >>>> Thanks >>>> Scott >>>> >>>> On 20/11/2009, at 1:08 AM, Christian Geisert wrote: >>>> >>>>> Scott Gray schrieb: >>>>>> It took a while but all our JUnit tests now >>> pass, next I'd like to get a continuous integration server >>> set up. Can anybody recommend some tools >>>>> >>>>> Why not just use what is available at the ASF ;-) >>>>> >>>>> http://ci.apache.org/ >>>>> >>>>> I use Hudson and like it, but I haven't tried the >>> others... >>>>> >>>>>> Once that is done I'd like to have it run the >>> tests after every commit and report any failures to the dev >>> list. It would then be the offending committer's >>> responsibility (well primarily at least) to fix the problem >>> as soon as possible, much like a build failure. If we >>> can't get everyone to agree to take on that responsibility >>> then I might as well stop now because I'll be damned if I'm >>> going to spend any more time fixing tests that I didn't >>> break :-) >>>>> >>>>> +1 >>>>> >>>>>> All of this should help us increase the >>> stability of the trunk and our confidence when taking a >>> checkout that we won't half to spend our development time >>> fixing things that used to work. It'll hopefully also >>> encourage the community to contribute more tests with the >>> knowledge that doing so will increase the stability of the >>> functionality they depend on. Fix a bug and you're >>> good for a day, write a test and you're good a lifetime :-) >>>>> >>>>> Absolutly agreed, big thanks for your (and all the >>> others) work in this area! >>>>> >>>>> --Christian >>>>> >>>> >>> >>> >> >> >> > |
| Free forum by Nabble | Edit this page |