Orocos log4cpp logging roadmap

Hello,

I continued my experiments with the new LoggingService and it's working
really fine. The deployer grouping method:

deployer-gnulinux -s deployLogging.xml -s deployAnythingElse.xml

really did the trick! I'm now able to log even in the component
constructor.

However, there's definitely some things that I would have to improve to make
it usable in my projects:

1. Possibility to set multiple appenders

The last talk about that was about the fact that the OCL::Category had only
one output port. Please tell me if I'm wrong, but we could just connect many
appenders (readers) to the category (writer) ? When writing a LoggingEvent,
it should be delivered to each reader. So, as far as I know, implementing
this feature is just a matter of supporting multiple xml appender statements
in the configuration file and modifying a bit the LoggingService.

2. Possibility to use networked appenders

If the feature above is implemented, it would be great to use one local
appender and then connect an additionnal appender over corba that could
merge all logs coming from different nodes.

3. Possibility to provide additionnal appenders [optional]

Actually, the ocl::logging supports some appenders, but it's impossible to
add custom appender and support them through the configuration file. I see
no simple way of doing that right now and I would like to have your opinion
about that.

I'm willing to participate to the development of these feature, but I don't
want to duplicate any work on that. Is there any pending tasks about that?
Is there a roadmap concerning this logging feature for futur releases of
RTT/OCL?

Philippe

Orocos log4cpp logging roadmap

On Jan 25, 2011, at 10:29 , Philippe Hamelin wrote:

> Hello,
>
> I continued my experiments with the new LoggingService and it's working really fine. The deployer grouping method:
>
> deployer-gnulinux -s deployLogging.xml -s deployAnythingElse.xml
>
> really did the trick! I'm now able to log even in the component constructor.

The group loading really did solve a how bunch of problems like this. Glad it works for you!

> However, there's definitely some things that I would have to improve to make it usable in my projects:
>
>
> 1. Possibility to set multiple appenders
>
> The last talk about that was about the fact that the OCL::Category had only one output port. Please tell me if I'm wrong, but we could just connect many appenders (readers) to the category (writer) ? When writing a LoggingEvent, it should be delivered to each reader. So, as far as I know, implementing this feature is just a matter of supporting multiple xml appender statements in the configuration file and modifying a bit the LoggingService.
>
> 2. Possibility to use networked appenders
>
> If the feature above is implemented, it would be great to use one local appender and then connect an additionnal appender over corba that could merge all logs coming from different nodes.
>
> 3. Possibility to provide additionnal appenders [optional]
>
> Actually, the ocl::logging supports some appenders, but it's impossible to add custom appender and support them through the configuration file. I see no simple way of doing that right now and I would like to have your opinion about that.
>
>
> I'm willing to participate to the development of these feature, but I don't want to duplicate any work on that. Is there any pending tasks about that? Is there a roadmap concerning this logging feature for futur releases of RTT/OCL?

Beyond the needs of my customers, I have been waiting for feedback from other Orocos users before deciding what to tackle next. The above is a good starting list.

I think 2 is already supported, you just need to create a NetworkedAppender component, and deploy that instead of the FileAppender that is currently there.

Re 3, the current model is one appender per component. So I think that 3 is already solved, as it is the more general case of 2. Just deploy a different component (which is the appender). Doesn't this solve your problem?

I see a couple of larger changes required

- 1 above, supporting multiple appenders per component. I'm not sure if your multiple readers on a buffered writer works as you say (Peter?).

- using log4cpp for _all_ RTT logging, which means tying together the RTT::Logger and log4cpp. I have a version of this, but it really has some issues. It works, but needs more consideration of the application level problems, and how these loggers mix together, particularly when RTT logs but before log4cpp is setup, making RTT use real-time logging (we haven't bothered to solve this yet), etc. This requires potentially system-wide changes to RTT, to prevent it using std::string in lots of places that it currently does.

- use of log4cpp from scripts and state machines. We've not yet tackled this, and there is an existing bug preventing this right now. You can't instantiate an rt_string in scripting or state machines in real-time, as RTT tries to make it a std::string instead. I don't believe this has been solved.

- runtime configuration of logging levels, etc. You should be able to do this from scripts, via services, etc. As we aren't using v2.x, we haven't started down this path.

Thoughts?
S

Orocos log4cpp logging roadmap

2011/1/26 S Roderick <kiwi [dot] net [..] ...>

> On Jan 25, 2011, at 10:29 , Philippe Hamelin wrote:
>
> > Hello,
> >
> > I continued my experiments with the new LoggingService and it's working
> really fine. The deployer grouping method:
> >
> > deployer-gnulinux -s deployLogging.xml -s deployAnythingElse.xml
> >
> > really did the trick! I'm now able to log even in the component
> constructor.
>
> The group loading really did solve a how bunch of problems like this. Glad
> it works for you!
>
> > However, there's definitely some things that I would have to improve to
> make it usable in my projects:
> >
> >
> > 1. Possibility to set multiple appenders
> >
> > The last talk about that was about the fact that the OCL::Category had
> only one output port. Please tell me if I'm wrong, but we could just connect
> many appenders (readers) to the category (writer) ? When writing a
> LoggingEvent, it should be delivered to each reader. So, as far as I know,
> implementing this feature is just a matter of supporting multiple xml
> appender statements in the configuration file and modifying a bit the
> LoggingService.
> >
> > 2. Possibility to use networked appenders
> >
> > If the feature above is implemented, it would be great to use one local
> appender and then connect an additionnal appender over corba that could
> merge all logs coming from different nodes.
> >
> > 3. Possibility to provide additionnal appenders [optional]
> >
> > Actually, the ocl::logging supports some appenders, but it's impossible
> to add custom appender and support them through the configuration file. I
> see no simple way of doing that right now and I would like to have your
> opinion about that.
> >
> >
> > I'm willing to participate to the development of these feature, but I
> don't want to duplicate any work on that. Is there any pending tasks about
> that? Is there a roadmap concerning this logging feature for futur releases
> of RTT/OCL?
>
> Beyond the needs of my customers, I have been waiting for feedback from
> other Orocos users before deciding what to tackle next. The above is a good
> starting list.
>
> I think 2 is already supported, you just need to create a NetworkedAppender
> component, and deploy that instead of the FileAppender that is currently
> there.
>
> Re 3, the current model is one appender per component. So I think that 3 is
> already solved, as it is the more general case of 2. Just deploy a different
> component (which is the appender). Doesn't this solve your problem?
>
>
That's right. I misread the LoggingService code and I had
not noticed that the Appenders were deployed by the user directly. So 2 and
3 above are already solved!

> I see a couple of larger changes required
>
> - 1 above, supporting multiple appenders per component. I'm not sure if
> your multiple readers on a buffered writer works as you say (Peter?).
>
>
If the buffer is on the writer side, that won't be that easy, because we
would have to dynamically create logging ports in the category.

> - using log4cpp for _all_ RTT logging, which means tying together the
> RTT::Logger and log4cpp. I have a version of this, but it really has some
> issues. It works, but needs more consideration of the application level
> problems, and how these loggers mix together, particularly when RTT logs but
> before log4cpp is setup, making RTT use real-time logging (we haven't
> bothered to solve this yet), etc. This requires potentially system-wide
> changes to RTT, to prevent it using std::string in lots of places that it
> currently does.
>
>
I think the
priority is rather centralized logs. Anyway, RTT:: Logger is already
notrealtime.

> - use of log4cpp from scripts and state machines. We've not yet tackled
> this, and there is an existing bug preventing this right now. You can't
> instantiate an rt_string in scripting or state machines in real-time, as RTT
> tries to make it a std::string instead. I don't believe this has been
> solved.
>
> - runtime configuration of logging levels, etc. You should be able to do
> this from scripts, via services, etc. As we aren't using v2.x, we haven't
> started down this path.
>
> Thoughts?
>

The last thing I didn't mention is the last post was about real-time string
stream. I modified RTT/OCL to support streaming to real-time categories such
as:

my_rt_cat.getStream(log4cpp::Priority::INFO) << "Here is the content of my
member variable: " << m_var;

That was pretty easy with the following typedef:

typedef std::basic_ostringstream<char, std::char_traits RTT::os::rt_allocator<char> > rt_ostringstream;

The change is not yet clean enough to me, but soon I will submit a patch for
this.

Philippe Hamelin