OCL::Logging to Syslog

OCL::Logging uses log4cpp under the covers, and log4cpp supports appending to syslog. The log4cpp dir inside orocos_toolchain has a (Remote)SyslogAppender class defined, but the OCL::Logging only appends File and Ostream.

Is syslog support planned / someone already working on it? Is anyone else interested in this capability? Pitfalls to watch for if we start to implement it?

--
Dustin Gooding
NASA/JSC Robotics
IM: dgooding [..] ...<xmpp:dgooding [..] ...>

OCL::Logging to Syslog

On 08/01/2013 04:14 PM, Gooding, Dustin R. (JSC-ER411) wrote:
> OCL::Logging uses log4cpp under the covers, and log4cpp supports
> appending to syslog. The log4cpp dir inside orocos_toolchain has a
> (Remote)SyslogAppender class defined, but the OCL::Logging only
> appends File and Ostream.
>
> Is syslog support planned / someone already working on it? Is anyone
> else interested in this capability? Pitfalls to watch for if we start
> to implement it?
We (Rock) are currently thinking about the best workflow to feed the
component's text output "somewhere". We realized that the best way is to
be able to feed all the text log (not only the stuff that goes over the
RTT logger) to a service (syslog or something else). We first thought
about syslog, but the issue is that it then gets into a system-wide
place ... Not optimal.

A colleague of mind was looking into this. He found
http://logstash.net/, and more importantly a set of "shippers" that
allow to tail-read text files and send them to various services (a lot
of them go over redis). We might go down that route.

I would be interested in having your opinion on this ...

OCL::Logging to Syslog

On 08/05/2013 02:11 PM, S Roderick wrote:
>> Different project members can have different test environments / runs /
>> experiments, and I have a bit of a problem with having all that stuff
>> going to system-wide logfiles / log systems. I have to admit that I am
>> not familiar with the "syslog workflow", so that might be a bad
>> preconceived impression on my side
> I'm with Sylvain on this one, particularly in situations where multiple users can run on the same machine. Also, syslog outputs aren't available to all users in all architectures. I don't want to have to give sudo rights to all users if there is an alternative ...
>
> If there are significant structural or design issues in the current logging implementation, maybe we can talk about those and see what we can do to address them. Adding syslog support to the current implementation should be fairly trivial ...
>
> I do wonder how logstash would work under load. It'd be interesting to see how it handles events coming at it at several hundred Hz, maybe from multiple sources simultaneously. Depends on your requirements I guess ...
Definitely. This is a major difference between you guys and text logging
in rock. If we have to generate samples at high frequency, we use ports
and Rock's logger, because it gives us a type, structured output with a
very low overhead. Text log integration is in our case for library text
output in which a dramatic performance drop is expected when enabling
DEBUG logging.

OCL::Logging to Syslog

On Aug 5, 2013, at 08:36 , Sylvain Joyeux <sylvain [dot] joyeux [..] ...> wrote:

> On 08/05/2013 02:11 PM, S Roderick wrote:
>>> Different project members can have different test environments / runs /
>>> experiments, and I have a bit of a problem with having all that stuff
>>> going to system-wide logfiles / log systems. I have to admit that I am
>>> not familiar with the "syslog workflow", so that might be a bad
>>> preconceived impression on my side
>> I'm with Sylvain on this one, particularly in situations where multiple users can run on the same machine. Also, syslog outputs aren't available to all users in all architectures. I don't want to have to give sudo rights to all users if there is an alternative ...
>>
>> If there are significant structural or design issues in the current logging implementation, maybe we can talk about those and see what we can do to address them. Adding syslog support to the current implementation should be fairly trivial ...
>>
>> I do wonder how logstash would work under load. It'd be interesting to see how it handles events coming at it at several hundred Hz, maybe from multiple sources simultaneously. Depends on your requirements I guess ...
> Definitely. This is a major difference between you guys and text logging
> in rock. If we have to generate samples at high frequency, we use ports
> and Rock's logger, because it gives us a type, structured output with a
> very low overhead. Text log integration is in our case for library text
> output in which a dramatic performance drop is expected when enabling
> DEBUG logging.

Yeah, we're going to change over to netcdf or something for storing the high frequency stuff (when we find a system level solution that we're comfortable with). But even with trace logging (vs high freq. sampling) we still operate in realtime under DEBUG logging. I'm not saying that the system isn't working hard at times ...
S

OCL::Logging to Syslog

On Aug 1, 2013, at 10:10 AM, Sylvain Joyeux <sylvain [dot] joyeux [..] ...sylvain [dot] joyeux [..] ...>> wrote:

On 08/01/2013 04:14 PM, Gooding, Dustin R. (JSC-ER411) wrote:
OCL::Logging uses log4cpp under the covers, and log4cpp supports appending to syslog. The log4cpp dir inside orocos_toolchain has a (Remote)SyslogAppender class defined, but the OCL::Logging only appends File and Ostream.

Is syslog support planned / someone already working on it? Is anyone else interested in this capability? Pitfalls to watch for if we start to implement it?
We (Rock) are currently thinking about the best workflow to feed the component's text output "somewhere". We realized that the best way is to be able to feed all the text log (not only the stuff that goes over the RTT logger) to a service (syslog or something else). We first thought about syslog, but the issue is that it then gets into a system-wide place ... Not optimal.

Couldn't you use the syslog facility to sort/organize the log messages? Yes, there will be a central location (local or remote), but it doesn't have to be messy. Give your robot(s) logs a special facility (based on category?) and use syslog.conf to route those specific log entires to the appropriate file.

Or do you have a different concern related to the system-wide place?

A colleague of mind was looking into this. He found http://logstash.net/, and more importantly a set of "shippers" that allow to tail-read text files and send them to various services (a lot of them go over redis). We might go down that route.

Fancy! Some of those shippers look promising. Syslog is still appealing to me for the sole sake that it's such a standard, and it's already running on our robot. But, I can understand the appeal of the logstash+shippers, especially since Orocos wouldn't *necessarily* have to start supporting syslog logging.

--
dustin

I would be interested in having your opinion on this ...

--
Sylvain Joyeux (Dr.Ing.)
Senior Researcher

Space & Security Robotics
Underwater Robotics

!!! Achtung, neue Telefonnummer!!!

Standort Bremen:
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Straße 5
28359 Bremen, Germany

Phone: +49 (0)421 178-454136
Fax: +49 (0)421 218-454150
E-Mail: robotik [..] ...<mailto:robotik [..] ...>

Weitere Informationen: http://www.dfki.de/robotik
-----------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Straße 122, D-67663 Kaiserslautern
Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster
(Vorsitzender) Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973
Steuernummer: 19/673/0060/3
-----------------------------------------------------------------------

OCL::Logging to Syslog

On 08/01/2013 05:37 PM, Gooding, Dustin R. (JSC-ER411) wrote:
> Couldn't you use the syslog facility to sort/organize the log
> messages? Yes, there will be a central location (local or remote),
> but it doesn't have to be messy. Give your robot(s) logs a special
> facility (based on category?) and use syslog.conf to route those
> specific log entires to the appropriate file.
>
> Or do you have a different concern related to the system-wide place?
Different project members can have different test environments / runs /
experiments, and I have a bit of a problem with having all that stuff
going to system-wide logfiles / log systems. I have to admit that I am
not familiar with the "syslog workflow", so that might be a bad
preconceived impression on my side

OCL::Logging to Syslog

On Aug 5, 2013, at 07:55 , Sylvain Joyeux <sylvain [dot] joyeux [..] ...> wrote:

> On 08/01/2013 05:37 PM, Gooding, Dustin R. (JSC-ER411) wrote:
>> Couldn't you use the syslog facility to sort/organize the log
>> messages? Yes, there will be a central location (local or remote),
>> but it doesn't have to be messy. Give your robot(s) logs a special
>> facility (based on category?) and use syslog.conf to route those
>> specific log entires to the appropriate file.
>>
>> Or do you have a different concern related to the system-wide place?
> Different project members can have different test environments / runs /
> experiments, and I have a bit of a problem with having all that stuff
> going to system-wide logfiles / log systems. I have to admit that I am
> not familiar with the "syslog workflow", so that might be a bad
> preconceived impression on my side

I'm with Sylvain on this one, particularly in situations where multiple users can run on the same machine. Also, syslog outputs aren't available to all users in all architectures. I don't want to have to give sudo rights to all users if there is an alternative ...

If there are significant structural or design issues in the current logging implementation, maybe we can talk about those and see what we can do to address them. Adding syslog support to the current implementation should be fairly trivial ...

I do wonder how logstash would work under load. It'd be interesting to see how it handles events coming at it at several hundred Hz, maybe from multiple sources simultaneously. Depends on your requirements I guess ...
S