Yarp threads and C++11 mutex

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Yarp threads and C++11 mutex

Truong Nghiem
Hello,

I am using the current version of Yarp on Mac OS, installed with homebrew.  I am developing a program in which there are multiple Yarp ports receiving inputs (for which I use callback method onRead() and set useCallback()). The onRead() callback method processes the received inputs and pushes them to a shared queue, managed and to be processed by another thread (let's call it the "main thread"). As I understand, the onRead() method is called in Yarp threads associated with the ports. The main thread and the shared queue use the C++11 standard (std::) thread, mutex, and condition variable. I have tested the main thread and the shared queue very carefully without Yarp, and they work fine. However, when I run my program with Yarp, the first call to std::mutex lock from a Yarp thread always cause a segmentation fault. I spent two hours inspecting the problem carefully and couldn't find any problem in my program, except the fact that I mix Yarp threads and standard mutex/thread. So I suspect the culprit is that Yarp doesn't work with C++11 mutex/thread.

My questions are:

1) Is my conclusion correct? Can't I mix Yarp threads and C++11 mutex/thread?

2) How should I solve this issue? I don't want to rewrite my code to use Yarp thread and mutex because I have invested heavily in my current code, and there are many other parts / libraries of my program which already use the standard thread / mutex. I figure that maybe I will need to create std::thread's in which I manually poll the Yarp ports and bypass the callback mechanism of Yarp. However, this is inefficient in terms of resource usage because, I think, Yarp already creates many threads for its ports. I also see an experimental option in Yarp called YARP_EXPERIMENTAL_CXX11, which is OFF by default. Might turning on that option help to solve my problem?

Thank you in advance!
Tim
 

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
_______________________________________________
Robotcub-hackers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/robotcub-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Yarp threads and C++11 mutex

Lorenzo Natale-2

Dear Tim,

can share the code that triggers this problem? Is it simple enough for us to look? We could check to see if you are using YARP in some way which can cause troubles.

 

Lorenzo

 

From: Truong Nghiem [mailto:[hidden email]]
Sent: mercoledì 18 febbraio 2015 09:40
To: [hidden email]
Subject: [rc-hackers] Yarp threads and C++11 mutex

 

Hello,

 

I am using the current version of Yarp on Mac OS, installed with homebrew.  I am developing a program in which there are multiple Yarp ports receiving inputs (for which I use callback method onRead() and set useCallback()). The onRead() callback method processes the received inputs and pushes them to a shared queue, managed and to be processed by another thread (let's call it the "main thread"). As I understand, the onRead() method is called in Yarp threads associated with the ports. The main thread and the shared queue use the C++11 standard (std::) thread, mutex, and condition variable. I have tested the main thread and the shared queue very carefully without Yarp, and they work fine. However, when I run my program with Yarp, the first call to std::mutex lock from a Yarp thread always cause a segmentation fault. I spent two hours inspecting the problem carefully and couldn't find any problem in my program, except the fact that I mix Yarp threads and standard mutex/thread. So I suspect the culprit is that Yarp doesn't work with C++11 mutex/thread.

 

My questions are:

 

1) Is my conclusion correct? Can't I mix Yarp threads and C++11 mutex/thread?

 

2) How should I solve this issue? I don't want to rewrite my code to use Yarp thread and mutex because I have invested heavily in my current code, and there are many other parts / libraries of my program which already use the standard thread / mutex. I figure that maybe I will need to create std::thread's in which I manually poll the Yarp ports and bypass the callback mechanism of Yarp. However, this is inefficient in terms of resource usage because, I think, Yarp already creates many threads for its ports. I also see an experimental option in Yarp called YARP_EXPERIMENTAL_CXX11, which is OFF by default. Might turning on that option help to solve my problem?

 

Thank you in advance!

Tim

 


------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
_______________________________________________
Robotcub-hackers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/robotcub-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Yarp threads and C++11 mutex

Truong Nghiem
Hi Lorenzo,

The code base is quite large and not open-source (yet), so I can't share it with you. However, the Yarp usage is quite simple and similar to the following description:

- A thread-safe shared queue is implemented in C++11, similar to this one (Implementing a Thread-Safe Queue using Condition Variables (Updated) | Just Software Solutions - Custom Software Development). The main thread just waits for data to be pushed to the queue, then pops and processes it.

- A buffered Yarp port is opened to receive incoming messages. The port uses callback method onRead() to process the message, convert it to a queue item which is then pushed to the shared queue. Note that the push() method uses C++11 mutex to lock the access to the queue.

When I run my program, the first incoming message to the Yarp port triggers the callback onRead() in the Yarp's thread, which calls push() of the shared queue, which calls unique_lock<mutex> of C++11 and the segmentation fault occurs right there.

Some updates:

- I tried to build Yarp on Mac OS with the flag YARP_EXPERIMENTAL_CXX11=TRUE but the compilation failed. I couldn't fix the error, so I rebuilt Yarp without the option.

- I solved the segmentation fault problem by manually creating my own communication thread. Basically, I disable Yarp's callback, and create a std::thread which polls the Yarp port for incoming messages and pushes data to the queue from that std::thread.  Everything works now. The drawback is that I will have an extra thread for each Yarp port, and my program will have many ports.

- So my conclusion is that, at the current state, I should not mix Yarp threads and C++11 thread / mutex.  Perhaps in the future, if Yarp will use C++11 thread support, we can mix them.

Best,
Tim


On Thursday, February 19, 2015 11:32 AM, Lorenzo Natale <[hidden email]> wrote:


Dear Tim,
can share the code that triggers this problem? Is it simple enough for us to look? We could check to see if you are using YARP in some way which can cause troubles.
 
Lorenzo
 
From: Truong Nghiem [mailto:[hidden email]]
Sent: mercoledì 18 febbraio 2015 09:40
To: [hidden email]
Subject: [rc-hackers] Yarp threads and C++11 mutex
 
Hello,
 
I am using the current version of Yarp on Mac OS, installed with homebrew.  I am developing a program in which there are multiple Yarp ports receiving inputs (for which I use callback method onRead() and set useCallback()). The onRead() callback method processes the received inputs and pushes them to a shared queue, managed and to be processed by another thread (let's call it the "main thread"). As I understand, the onRead() method is called in Yarp threads associated with the ports. The main thread and the shared queue use the C++11 standard (std::) thread, mutex, and condition variable. I have tested the main thread and the shared queue very carefully without Yarp, and they work fine. However, when I run my program with Yarp, the first call to std::mutex lock from a Yarp thread always cause a segmentation fault. I spent two hours inspecting the problem carefully and couldn't find any problem in my program, except the fact that I mix Yarp threads and standard mutex/thread. So I suspect the culprit is that Yarp doesn't work with C++11 mutex/thread.
 
My questions are:
 
1) Is my conclusion correct? Can't I mix Yarp threads and C++11 mutex/thread?
 
2) How should I solve this issue? I don't want to rewrite my code to use Yarp thread and mutex because I have invested heavily in my current code, and there are many other parts / libraries of my program which already use the standard thread / mutex. I figure that maybe I will need to create std::thread's in which I manually poll the Yarp ports and bypass the callback mechanism of Yarp. However, this is inefficient in terms of resource usage because, I think, Yarp already creates many threads for its ports. I also see an experimental option in Yarp called YARP_EXPERIMENTAL_CXX11, which is OFF by default. Might turning on that option help to solve my problem?
 
Thank you in advance!
Tim
 



------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
_______________________________________________
Robotcub-hackers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/robotcub-hackers
Loading...