[orocosusers] issue with Xenomai + soem master

Hi all,

I have problems with the soem master under xenomai.

I will grateful to any one that will help in getting some hint about where to start.

The slave is the ethercat NI9144

Everything works fine for several minutes, and then the slave goes in error, and i get the following errors

/home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(warn_upon_switch+0x16)[0x7efee29d35e6]
/lib/x86_64-linux-gnu/libc.so.6(+0x36cb0)[0x7efee0d85cb0]
[0x7ffe7ab5e9cb]
/lib/x86_64-linux-gnu/libc.so.6(__clock_gettime+0x1d)[0x7efee0e5b4dd]
/opt/ros/indigo/lib/libosal.so(osal_gettimeofday+0x15)[0x7efeab51e895]
/opt/ros/indigo/lib/libosal.so(osal_timer_start+0x15)[0x7efeab51e915]
/opt/ros/indigo/lib/liboshw.so(ecx_waitinframe+0x1d)[0x7efeab31bced]
/opt/ros/indigo/lib/libsoem.so(ecx_receive_processdata_group+0x67)[0x7efeb0104c17]
/home/eec/ws/catkin_ws/devel/lib/orocos/xenomai/soem_master/libsoem_master-xenomai.so(_ZN11soem_master19SoemMasterComponent10updateHookEv+0x48)[0x7efebc23e3ac]
/home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT15ExecutionEngine15processChildrenEv+0x1aa)[0x7efee29c435a]
/home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT2os15thread_functionEPv+0x164)[0x7efee29ce1f4]
/home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT2os24rtos_xeno_thread_wrapperEPv+0x24)[0x7efee29d3634]
/usr/xenomai/lib/libnative.so.3(+0x46a9)[0x7efee193c6a9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7efee0518184]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7efee0e4cffd]

i am using kernel 3.18.20-xenomai- 2.6.5

The system has some strange behavior regarding the keyboard, as it produce some time the same keyboard stroke several times.

thanks, Gianni.

------------------------------------------------
Gianni Borghesan, PhD
Robot Assisted Surgery group
Robotics Research group
KU Leuven,
Department of Mechanical Engineering,
Division of PMA
Room 02.019
Celestijnenlaan 300B, B-3001 Heverlee, Belgium.
Tel: +32 16 322515.
Fax: +32 16 322987
------------------------------------------------

[orocosusers] issue with Xenomai + soem master

Hi Gianni,

Has this already been solved?

Question. Do the tests of SOEM library run without problem?

I think these _ZN3 0X sort of problems are usually caused by missing links
and dependencies...

I don't see a mention of xenomai on OpenEtherCATsociety.github.io.

Regards
Keivan

On 24 October 2017 at 15:53, Gianni Borghesan <gianni [dot] borghesan [..] ...>
wrote:

> Hi all,
>
>
> I have problems with the soem master under xenomai.
>
> I will grateful to any one that will help in getting some hint about where
> to start.
>
> The slave is the ethercat NI9144
>
> Everything works fine for several minutes, and then the slave goes in
> error, and i get the following errors
>
>
> /home/eec/ws/underlay_isolated/install_isolated/lib/
> liborocos-rtt-xenomai.so.2.8(warn_upon_switch+0x16)[0x7efee29d35e6]
> /lib/x86_64-linux-gnu/libc.so.6(+0x36cb0)[0x7efee0d85cb0]
> [0x7ffe7ab5e9cb]
> /lib/x86_64-linux-gnu/libc.so.6(__clock_gettime+0x1d)[0x7efee0e5b4dd]
> /opt/ros/indigo/lib/libosal.so(osal_gettimeofday+0x15)[0x7efeab51e895]
> /opt/ros/indigo/lib/libosal.so(osal_timer_start+0x15)[0x7efeab51e915]
> /opt/ros/indigo/lib/liboshw.so(ecx_waitinframe+0x1d)[0x7efeab31bced]
> /opt/ros/indigo/lib/libsoem.so(ecx_receive_processdata_
> group+0x67)[0x7efeb0104c17]
> /home/eec/ws/catkin_ws/devel/lib/orocos/xenomai/soem_
> master/libsoem_master-xenomai.so(_ZN11soem_master19SoemMasterComponent10u
> pdateHookEv+0x48)[0x7efebc23e3ac]
> /home/eec/ws/underlay_isolated/install_isolated/lib/
> liborocos-rtt-xenomai.so.2.8(_ZN3RTT15ExecutionEngine15proce
> ssChildrenEv+0x1aa)[0x7efee29c435a]
> /home/eec/ws/underlay_isolated/install_isolated/lib/
> liborocos-rtt-xenomai.so.2.8(_ZN3RTT2os15thread_functionEPv+
> 0x164)[0x7efee29ce1f4]
> /home/eec/ws/underlay_isolated/install_isolated/lib/
> liborocos-rtt-xenomai.so.2.8(_ZN3RTT2os24rtos_xeno_thread_
> wrapperEPv+0x24)[0x7efee29d3634]
> /usr/xenomai/lib/libnative.so.3(+0x46a9)[0x7efee193c6a9]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7efee0518184]
> /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7efee0e4cffd]
>
>
> i am using kernel 3.18.20-xenomai- 2.6.5
>
>
> The system has some strange behavior regarding the keyboard, as it produce
> some time the same keyboard stroke several times.
>
>
> thanks, Gianni.
>
>
>
> ------------------------------------------------
> Gianni Borghesan, PhD
> Robot Assisted Surgery group
> Robotics Research group
> KU Leuven,
> Department of Mechanical Engineering,
> Division of PMA
> Room 02.019
> Celestijnenlaan 300B
> <https://maps.google.com/?q=Celestijnenlaan+300B&entry=gmail&source=g>,
> B-3001 Heverlee, Belgium.
> Tel: +32 16 322515 <016%2032%2025%2015>.
> Fax: +32 16 322987 <016%2032%2029%2087>
> ------------------------------------------------
>

Ruben Smits's picture

[orocosusers] issue with Xenomai + soem master

Reading the error from top to bottom:

* It is giving you a warn_upon_switch error, which means that xenomai is
switching from HRT to SRT for some reason.
* My first thought would be that is not HRT safe to call __clock_gettime__.

So if I'm not mistaken you will need to patch SOEM for xenomai to replace
that __clock_gettime__ or osal_timer_start with a Xenomai HRT safe
alternative.

My 2 cents,

Ruben

On Wed, Oct 25, 2017 at 3:23 PM Keivan Zavari <keivan [dot] zavari [..] ...>
wrote:

> Hi Gianni,
>
> Has this already been solved?
>
> Question. Do the tests of SOEM library run without problem?
>
> I think these _ZN3 0X sort of problems are usually caused by missing links
> and dependencies...
>
> I don't see a mention of xenomai on OpenEtherCATsociety.github.io.
>
>
> Regards
> Keivan
>
>
> On 24 October 2017 at 15:53, Gianni Borghesan <
> gianni [dot] borghesan [..] ...> wrote:
>
>> Hi all,
>>
>>
>> I have problems with the soem master under xenomai.
>>
>> I will grateful to any one that will help in getting some hint about
>> where to start.
>>
>> The slave is the ethercat NI9144
>>
>> Everything works fine for several minutes, and then the slave goes in
>> error, and i get the following errors
>>
>>
>>
>> /home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(warn_upon_switch+0x16)[0x7efee29d35e6]
>> /lib/x86_64-linux-gnu/libc.so.6(+0x36cb0)[0x7efee0d85cb0]
>> [0x7ffe7ab5e9cb]
>> /lib/x86_64-linux-gnu/libc.so.6(__clock_gettime+0x1d)[0x7efee0e5b4dd]
>> /opt/ros/indigo/lib/libosal.so(osal_gettimeofday+0x15)[0x7efeab51e895]
>> /opt/ros/indigo/lib/libosal.so(osal_timer_start+0x15)[0x7efeab51e915]
>> /opt/ros/indigo/lib/liboshw.so(ecx_waitinframe+0x1d)[0x7efeab31bced]
>>
>> /opt/ros/indigo/lib/libsoem.so(ecx_receive_processdata_group+0x67)[0x7efeb0104c17]
>>
>> /home/eec/ws/catkin_ws/devel/lib/orocos/xenomai/soem_master/libsoem_master-xenomai.so(_ZN11soem_master19SoemMasterComponent10updateHookEv+0x48)[0x7efebc23e3ac]
>>
>> /home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT15ExecutionEngine15processChildrenEv+0x1aa)[0x7efee29c435a]
>>
>> /home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT2os15thread_functionEPv+0x164)[0x7efee29ce1f4]
>>
>> /home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT2os24rtos_xeno_thread_wrapperEPv+0x24)[0x7efee29d3634]
>> /usr/xenomai/lib/libnative.so.3(+0x46a9)[0x7efee193c6a9]
>> /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7efee0518184]
>> /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7efee0e4cffd]
>>
>>
>> i am using kernel 3.18.20-xenomai- 2.6.5
>>
>>
>> The system has some strange behavior regarding the keyboard, as it
>> produce some time the same keyboard stroke several times.
>>
>>
>> thanks, Gianni.
>>
>>
>>
>> ------------------------------------------------
>> Gianni Borghesan, PhD
>> Robot Assisted Surgery group
>> Robotics Research group
>> KU Leuven,
>> Department of Mechanical Engineering,
>> Division of PMA
>> Room 02.019
>> Celestijnenlaan 300B
>> <https://maps.google.com/?q=Celestijnenlaan+300B&entry=gmail&source=g>,
>> B-3001 Heverlee, Belgium.
>> Tel: +32 16 322515 <016%2032%2025%2015>.
>> Fax: +32 16 322987 <016%2032%2029%2087>
>> ------------------------------------------------
>>
>
>
>
> --
>
> Kind regards,
> Keivan Zavari
>
> Fazantenlaan 2,
> <https://maps.google.com/?q=Fazantenlaan+2,3050+Oud-Heverlee,+Belgium&entry=gmail&source=g>
> 3050 Oud-Heverlee,
> <https://maps.google.com/?q=Fazantenlaan+2,3050+Oud-Heverlee,+Belgium&entry=gmail&source=g>
> Belgium
> <https://maps.google.com/?q=Fazantenlaan+2,3050+Oud-Heverlee,+Belgium&entry=gmail&source=g>
>
> Mob : *+32 489 001 206*
>

[orocosusers] issue with Xenomai + soem master

Hi Ruben, Hi Kevain, thanks for the answers!

After writing I noticed that one other symptom was a big clock drift.

after some more digging, I found this thread:
https://xenomai.org/pipermail/xenomai/2015-February/033129.html

and I tried to change the clock source as indicated
(GRUB_CMDLINE_LINUX="clocksource=tsc")

This seems to solve the issue (both the clock drift and the soem master,
and all the others strange behaviours).

I run the system for several hours in a row without problems.

In conclusion the problem was not related to soem master - orocos, but a
more general problem.

To xenomai gurus: is this solution ok?

Cheers, Gianni

On 07/11/17 10:18, Ruben Smits wrote:
> AFAIU:
> Reading the error from top to bottom:
>
> * It is giving you a warn_upon_switch error, which means that xenomai
> is switching from HRT to SRT for some reason.
> * My first thought would be that is not HRT safe to call
> __clock_gettime__.
>
> So if I'm not mistaken you will need to patch SOEM for xenomai to
> replace that __clock_gettime__ or osal_timer_start with a Xenomai HRT
> safe alternative.
>
> My 2 cents,
>
> Ruben
>
> On Wed, Oct 25, 2017 at 3:23 PM Keivan Zavari <keivan [dot] zavari [..] ...
> <mailto:keivan [dot] zavari [..] ...>> wrote:
>
> Hi Gianni,
>
> Has this already been solved?
>
> Question. Do the tests of SOEM library run without problem?
>
> I think these _ZN3 0X sort of problems are usually caused by
> missing links and dependencies...
>
> I don't see a mention of xenomai on OpenEtherCATsociety.github.io
> <http://OpenEtherCATsociety.github.io>.
>
>
> Regards
> Keivan
>
>
> On 24 October 2017 at 15:53, Gianni Borghesan
> <gianni [dot] borghesan [..] ...
> <mailto:gianni [dot] borghesan [..] ...>> wrote:
>
> Hi all,
>
>
> I have problems with the soem master under xenomai.
>
> I will grateful to any one that will help in getting some hint
> about where to start.
>
> The slave is the ethercat NI9144
>
> Everything works fine for several minutes, and then the slave
> goes in error, and i get the following errors
>
>
> /home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(warn_upon_switch+0x16)[0x7efee29d35e6]
> /lib/x86_64-linux-gnu/libc.so.6(+0x36cb0)[0x7efee0d85cb0]
> [0x7ffe7ab5e9cb]
> /lib/x86_64-linux-gnu/libc.so.6(__clock_gettime+0x1d)[0x7efee0e5b4dd]
> /opt/ros/indigo/lib/libosal.so(osal_gettimeofday+0x15)[0x7efeab51e895]
> /opt/ros/indigo/lib/libosal.so(osal_timer_start+0x15)[0x7efeab51e915]
> /opt/ros/indigo/lib/liboshw.so(ecx_waitinframe+0x1d)[0x7efeab31bced]
> /opt/ros/indigo/lib/libsoem.so(ecx_receive_processdata_group+0x67)[0x7efeb0104c17]
> /home/eec/ws/catkin_ws/devel/lib/orocos/xenomai/soem_master/libsoem_master-xenomai.so(_ZN11soem_master19SoemMasterComponent10updateHookEv+0x48)[0x7efebc23e3ac]
> /home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT15ExecutionEngine15processChildrenEv+0x1aa)[0x7efee29c435a]
> /home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT2os15thread_functionEPv+0x164)[0x7efee29ce1f4]
> /home/eec/ws/underlay_isolated/install_isolated/lib/liborocos-rtt-xenomai.so.2.8(_ZN3RTT2os24rtos_xeno_thread_wrapperEPv+0x24)[0x7efee29d3634]
> /usr/xenomai/lib/libnative.so.3(+0x46a9)[0x7efee193c6a9]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7efee0518184]
> /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7efee0e4cffd]
>
>
>
> i am using kernel 3.18.20-xenomai- 2.6.5
>
>
> The system has some strange behavior regarding the keyboard,
> as it produce some time the same keyboard stroke several times.
>
>
> thanks, Gianni.
>
>
>
> ------------------------------------------------
> Gianni Borghesan, PhD
> Robot Assisted Surgery group
> Robotics Research group
> KU Leuven,
> Department of Mechanical Engineering,
> Division of PMA
> Room 02.019
> Celestijnenlaan 300B
> <https://maps.google.com/?q=Celestijnenlaan+300B&entry=gmail&source=g>,
> B-3001 Heverlee, Belgium.
> Tel: +32 16 322515 <tel:016%2032%2025%2015>.
> Fax: +32 16 322987 <tel:016%2032%2029%2087>
> ------------------------------------------------
>
>
>
>
> --
>
> Kind regards,
> Keivan Zavari
>
> Fazantenlaan 2,
> <https://maps.google.com/?q=Fazantenlaan+2,3050+Oud-Heverlee,+Belgium&entry=gmail&source=g>
> 3050 Oud-Heverlee,
> <https://maps.google.com/?q=Fazantenlaan+2,3050+Oud-Heverlee,+Belgium&entry=gmail&source=g>
> Belgium
> <https://maps.google.com/?q=Fazantenlaan+2,3050+Oud-Heverlee,+Belgium&entry=gmail&source=g>
>
> Mob : *+32 489 001 206*
>