On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

I love this idea too. It's "too complex" to go in the execution engine to
makes tests working with their tested components (and maybe not very
maintainable).

And what you proposed Peter also allows functionnal testing because you can
have your application deployed and then plug on it the tests, which can
leads to non-regression testing.

The main thing for the proposed design is to have it easy to write an deploy
for the user (because no one like writing unit tests so if it's complex to
do they are not written), and having tests reports (like boost xml reports)
in order to integrate this with buildbots like Hudson.

2010/10/18 Peter Soetens <peter [..] ...>

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
> > Hi all,
> >
> > I would like to set up unit testing for my project. As I saw that there
> is
> > Orocos Tests, I wonder if it exists any documentation or main classes
> that
> > I could use directly from rtt in my project. If none I will read rtt
> code,
> > but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only
> toggle
> a flag or so. The unit test then checks if the flag was toggled after using
> the
> functionality.
>
> So this is hardly what you need for unit testing an application component.
> The
> way I would unit test a component is by exercising its operations and ports
> in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
>
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
>
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
>
> // this program could be loaded in 'unit' too (or unit is a service in the
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the
> arg-"bob.isR..."
>
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not
> receive
> message")
>   // ...
> }
> 

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S

On Oct 18, 2010, at 11:21 , Peter Soetens wrote:

> On Sunday 17 October 2010 21:42:49 Willy Lambert wrote:
>> Hi all,
>>
>> I would like to set up unit testing for my project. As I saw that there is
>> Orocos Tests, I wonder if it exists any documentation or main classes that
>> I could use directly from rtt in my project. If none I will read rtt code,
>> but If I could avoid to read this from scratch it would save my time :)
>
> The only way we unit test our components in RTT is by creating 'mock-up'
> components, ie use some functionality with 'fake' functions, which only toggle
> a flag or so. The unit test then checks if the flag was toggled after using the
> functionality.
>
> So this is hardly what you need for unit testing an application component. The
> way I would unit test a component is by exercising its operations and ports in
> a script. We could build a unit test component, similar to a logging
> component, that provides the unit testing API the script needs.
>
> The script could then look like:
>
> (loaded in a deployer):
>

> // create components and ports
> program SetupTest {
>  import("unit") // loads the unit test component library
>  loadComponent("unit","unit") // creates the component
>  // OR loads it as a service:
>  loadService("unit","unit")
> 
>  import("myapp") // loads the component library to test
>  loadComponent("bob","myapp::FooBar") // creates one instance of FooBar
> 
>  unit.newOutputPort("A","myapp.FooBarStruct")
> }
> 
> // this program could be loaded in 'unit' too (or unit is a service in the 
> deployer):
> program TestSomeFeature {
>   bob.parameter="baz"
>   unit.Require( bob.isRunning() ) // we could also quote the arg-"bob.isR..."
> 
>   connectTwoPorts("unit","A","bob","B")
>   unit.A.write( myapp.FooBarStruct ) // write the port
>   unit.CheckMessage( bob.receiveCount() == 1, "myapp::FooBar did not receive 
> message")
>   // ...
> }
> 

To date we've done this by attaching a mockup-peer component that wraps the component to be tested, making sure all the port connections are made, etc. We then manually modify ports and call methods, and call execute() on both the mockup and the test component. We can verify outputs, errors, etc, this way, but it is quite tricky to get right. It is also very hard to do with components that require periodicity - the simulation activity just doesn't quite do it.

Definitely an area for Orocos 2.x or 3.x to put in some intensive work.
S