Monday, December 06, 2010

Writing a profiler for mono

As some of you may know, my day job is working on Moonlight, the FOSS implementation of Silverlight. This is a fairly complex piece of code as we have C++ interoperating with C# and Javascript, so managing the lifecycle of objects can be difficult. C++ refcounts, C# has an automatic garbage collector and Javascript... well, let's just say that it complicates things a *lot* when combined with the other two languages.

One of the tricks we use to ensure that objects do not die early is to use a (normal) GCHandle on our C# objects to ensure the garbage collector doesn't mark them as junk before they're definitely finished with in C++. However, we were having issues where some of our objects appeared to never ever get garbage collected even though they should be eligible. We suspected that excess GCHandles were causing it but we had no easy way of tracking this. Without detailed information on when and where the GCHandles were allocated, it would be next to impossible to track down where we were going wrong. After several aborted attempts to track this with Console.WriteLine I figured there must be a better way.

How about a custom profiler for mono?

Documentation on this is fairly sparse currently, so hopefully this will help. Thanks go to Paolo Molaro for helping me when I got stuck! This code is mostly copied/pasted from moonlight so it's C++ ish and also may contain some moonlight specific code.

Step 1:
Open up the "mono/metadata/profiler.h" file and check out all the profiler hooks that are available. They are all documented in profiler.c, so open that if you want a description of how they all work. Take note of the ones you'll need. All I needed to do was track GCHandle allocations, so I just wanted this hook:
void mono_profiler_install_gc_roots    (MonoProfileGCHandleFunc handle_callback, MonoProfileGCRootFunc roots_callback);


Step 2:
I now know what function I need to provide and the data I want to gather, so it's time to create a struct to store my data. This is an opaque struct which will be passed to the mono runtime and passed back in every profiler hook, so you can stick whatever you want in it. I know I want to store stacktraces, allocated gchandles and a typename, so I'll start with just those. I'll also declare my prototype for my profiler hook here.

struct _MonoProfiler {
const char *type_name;
GPtrArray *gchandles;
GPtrArray *stacktraces;
Moonlight::Mutex locker; /* used to ensure only one thread accesses the arrays */

static void track_gchandle (_MonoProfiler *prof, int op, int type, uintptr_t handle, MonoObject *obj);
};

typedef _MonoProfiler MonoProfiler;



Step 3:
Implementation time! First the constructor for the MonoProfiler. I use an environment variable to limit which types we store stacktraces for as they're expensive to generate and consume a lot of memory to store when you have 10,000s of them. The important functions are the last three. The first registers our profiler with the runtime. The second installs the hook so we can get notifications for gchandle allocations. The final call enables the gc_handle events in the profiler which makes the runtime invoke the hook we just registered.

_MonoProfiler::_MonoProfiler ()
{
type_name = g_getenv ("GCHANDLES_FOR_TYPE");

gchandles = g_ptr_array_new ();
stacktraces = g_ptr_array_new_with_free_func (g_free);
/* Register the profiler with mono */
mono_profiler_install (this, NULL);
/* Supply a function for the gc_roots hook */
mono_profiler_install_gc_roots (track_gchandle, NULL);
/* Enable gc_roots tracking in the profiler so our hook is invoked */
mono_profiler_set_events (MONO_PROFILE_GC_ROOTS);
}


Now all I need is the implementation for track_gchandle and I'm done!
void
MonoProfiler::track_gchandle (MonoProfiler *prof, int op, int type, uintptr_t handle, MonoObject *obj)
{
// Ignore anything that isn't a strong GC handle (docs say type == 2 is a strong gchandle)
if (type != 2)
return;

prof->locker.Lock ();

GPtrArray *gchandles = prof->gchandles;
GPtrArray *stacktraces = prof->stacktraces;

if (op == MONO_PROFILER_GC_HANDLE_CREATED) {
// Add the GCHandle to this array
g_ptr_array_add (gchandles, (gpointer) handle);
// If the target of the gchandle is of the correct type, store its stack trace
// Otherwise store NULL so that we can keep the index of the gchandle and corresponding
// stack trace in sync.
if (prof->type_name && !strcmp (prof->type_name, mono_class_get_name (mono_object_get_class(obj))))
g_ptr_array_add (stacktraces, get_stack_trace ());
else
g_ptr_array_add (stacktraces, NULL);
} else if (op == MONO_PROFILER_GC_HANDLE_DESTROYED) {
// Walk our list of gchandles and when we find the index of the destroyed handle
// remove the handle and corresponding stacktrace
for (int i = 0; i < (int)gchandles->len; i++) {
if (g_ptr_array_index (gchandles, i) == (gpointer) handle) {
g_ptr_array_remove_index_fast (gchandles, i);
g_ptr_array_remove_index_fast (stacktraces, i);
break;
}
}
}

prof->locker.Unlock ();
}


Step 4:
Profit! When you're ready, you can iterate over the gchandle array and pull out whatever statistics you want. For example here is the code I use to iterate the gchandles array and work out how many instances of each object type there are.
void
accumulate_g_ptr_array_by_type (gpointer data, gpointer user_data)
{
// The hashtable I passed in as the user_data in g_ptr_array_foreach
// which i am using to link a type name to a count
GHashTable *by_type = (GHashTable*) user_data;

// Get the MonoObject from the gchandle.
MonoObject *ob = mono_gchandle_get_target (GPOINTER_TO_INT (data));

// Get the type name from the mono_object
const char *name = mono_class_get_name (mono_object_get_class(ob));

// Find out how many instances we have already
int count = GPOINTER_TO_INT (g_hash_table_lookup (by_type, name)) + 1;

// Update the hashtable with an incremented count.
g_hash_table_insert (by_type, name, GINT_TO_POINTER (count));
}


I run that hashtable through another function to order to sort the list by the number of each type of object allocated so I can have a nicely formatted list as follows:

1 instances GCHandled of type Surface
1 instances GCHandled of type Deployment
4 instances GCHandled of type NameScope
20 instances GCHandled of type ControlTemplate
218 instances GCHandled of type MonoType
3985 instances GCHandled of type Uri


Needless to say, it's now immediately obvious to me that Uris are being GChandled and never freed, so i just have to enable tracing of their stacktraces and I can see exactly where we allocated the handle and from there figure out why we haven't freed it.

65 comments:

sports handicapping services said...

I wonder why I did not knew this before.... it was a good read. You are working on a very good blog.. I look forward to visit your blog again....

ADmin said...

What would you be able http://uk-essay.net/custom-essay to do to serve some useful purpose? The response is: a considerable amount.

dissertation writing said...

The codes are too difficult for me to understand please elaborate more. Exactly where perhaps you have emerged your own bottom line? I should be aware of factors that produced you're able to this theory? You may could reveal a few helpful pointers?

aliyaa said...

The eras personal statement length provides best writing opportunity for the readers and writers so that they can write perfect statement.

Unknown said...

Pretty good post! I just stumbled upon your blog and wanted coolessay.net reviews to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon.

huong duong said...

When it comes to thank the other person that you know why that is, they have to wait like and respect their time spent for you. Verbal apology not fully reflect that, further to say sorry too much will cause discomfort to the other
baixar musicas gratis , baixar snapchat , geometry dash 2.0 , play run 2 , b612

Faisal said...

Two full thumbs up for this magnificent article of yours. I've really enjoyed reading this article today.casio

KarlDeville said...

Thank you for taking the time and sharing this information with us. It was indeed very helpful and insightful while being straight forward and to the point.
mcdonaldsgutscheine.net/ | Startlr.com/ | SaludLimpia.com/

Anonymous said...

I want to buy Casio G Shock GAS-100BR-1A...!!
someone suggested this online store for buying my favourite Casio G Shock GAS-100BR-1A.Can anyone guide me this site is good or not...?

eve green said...

You have shared the information in such a way that i like it and wanted to share it on my wall
out door furniture nz and i was going to share it on my wall.

Freddi King said...

Really amazing blog, I’d love to discover some extra information.
SaludPulso.com

watson said...

It is great to have the opportunity to read a good quality article with useful information on topics that plenty are interested one. captain america civil war jacket I concur with your conclusions and will eagerly look forward to your future updates.

William Jacket said...

Thanks for sharing such information with us. It was too useful and productive. Unicorn Store Brie Larson Jacket . Please keep posting such great work.

Pankaj Singh said...

Keep it up for more information about this related blog. Visit OGEN Infosystem for the best web designing and development also get SEO Services in Delhi, India.
SEO Service in Delhi

Pradeep Appslure said...

Mobile app development company in gurgaon

Ruben Wilson said...

This is a really great post, I like Star wars jacket mens I am interested in reading this post.

Unknown said...

Thank you for sharing such an great article with us. I am very happy to read your article.
David Harbour Black Widow Cotton Jacket

shimaa shaker said...

لابد دائما من الاستمرار في تنظيف الخزانات كل 3 شهور لأن هذه يعمل على تكون البكتريا والفطريات ويعمل العديد من الامراض لذلك لابد من تنظيف الخزانات بدورية وعلى ايدى عامل ماهر وذلك ستجده في شركتنا شركة زهرة الخليج
تنظيف خزانات
شركة تنظيف خزانات بعجمان
شركة تنظيف خزانات بدبي
شركة تنظيف خزانات بالفجيرة
شركة تنظيف خزانات بالعين
شركة تنظيف خزانات بابوظبي
شركة تنظيف خزانات بالشارقة

Chicago P.D. Jon Seda Jackets said...

famousmoviejackets.com is now offering best quality and stylish Chicago P.D. Jon Seda Black Leather Jacket available with free shipping. For more celeb trends visit our store.

jeewangarg said...

Wow!!! It was really an Informational Article which provide me with much Insightful Information. I would like to first thank the author for such an Insightful Content. If someone here is looking for a Website Designing Company in India, then look no further than Jeewangarg.com as we are the Best website designing company in Delhi working in the arena for the last 8 years.
Digital Marketing company in Delhi

virgoacp said...

Wow!!! A great insightful & informative content on ACP Board. Virgo Group is the largest providers of best in class leading facade materials and more & having years of experience in manufacturing Premium ACP Sheet. We aim to deliver excellence and ensure maximum worth for our clients’ money and so we provide you with the best ACP Sheet Price.

silverservicecabs said...

Thank You for providing us with such an insightful information through this blog. However, if you are in the search of finding Cab or Taxi Services in Melbourne? Your Search Ends here…
silver service cabs is your answer for availing best silver taxi service in Melbourne. Reach us for the best Silver Service Cab Melbourne.

Servo said...

Thanks for the great article and post. Need a servo voltage stabilizer manufacturers in india? Don’t go beyond servo star “A popular 10 kva single phase voltage stabilizer which offers an extensive range of exceptional servo stabilizer. Feel Free to contact.

nexusups said...

Thanks for the great article and post. Need a Industrial UPS? Don’t go beyond nexusups. 10 kva online ups which offers an extensive range of exceptional Industrial UPS, and Online UPS Free to contact.

Silicon Valley jacket said...

This is a really great post, I like Silicon Valley jacket. I am interested in reading this post.

SBI Kiosk Banking said...


Thanks for sharing such a great information, Here we are India's best service provider of SBI Kiosk Banking.
kiosk banking
online sbi kiosk

rightselects said...

Wow!!! It was a great article I came to know a lot about the elevators by going through this article. If you’re also looking for a Hydraulic Passenger Lift, visit right select to gain the best possible information about the top 10 elevator companies in india.

phautecouture said...

I would first like to thank the author for coming up with the insightful information every time. People if you are also looking for the best indian dresses online | buy lehenga online then visit Panache Haute Couture, a leading Indian Designer House for Indian Dresses.

Essay Help Online said...

This blog is very nice.I really like such a fantastic written blog.I will keep coming here again and again.
Write My Essay UK

Kate said...

Experience the great style for less. Shop this premium quality striking Emilia Clarke Last Christmas Coat and enjoy free shipping in USA, UK and Canada.

Famous Movie Jackets said...

Don't miss this Birds Of Prey Harley Quinn Jacket to stay cool and classy this season. Available at Famous Movie Jackets at best budget. Get Yours!

Kate said...

Want to grab unique style statements? This Anne Hathaway Coat is all set to serve you dramatically shop from our online store for less.

Mike Lowrey jacket said...

Make an iconic appearance and get ready to steal the limelight gracefully wearing Mike Lowrey Cotton jacket at fair price from famousjacket.com

Mike Lowrey Cotton jacket said...

Feel the grace and indulgence of designer outfits. Shop Will Smith Mike Lowrey jacket at amazing discounted price with free shipping.

Bad Boys For Life Jacket said...

Experience a hypnotic look wearing sophisticated outfit inspired by Movie Bad Boys For LifeShop Bad Boys For Life Will Smith Jacket for 75% off.

3A-MOTO said...

Interesting post! Thank you for sharing them! I hope you will continue to have similar posts to share with everyone.Superhero Leather Jackets

Sam said...

For those students who are stuck with their College or University Assignments and are in desperate need of some professional assignment writing services, then My Assignment Help is the place you want to be because our experts have the knowledge, skills, and experience to help college and university students with their research papers, essays, case studies and Thesis projects

Mind Mingles said...

It’s a very interesting and knowledgeable post. Through this post, I learn many things. I appreciate your work and efforts. Thanks digital marketing packages , guest post services , seo reseller program , content writing prices , buy guest posts , seo services india, digital marketing agency in india , seo outsourcing India , app store seo

Mike Lowrey jacket said...

Don't miss the grand sale on Cotton Jacket and coat. Shop this Mike Lowrey jacket and avail flat 40% off on all the leather products with free shipping.

Bad Boys For Life Will Smith Jacket said...

Buy this premium quality Bad Boys For Life Will Smith Jacket from our online Fashion Hub filmsjacket. Shop and get up to 80% off on selected leather jackets and coats.

Mike Lowrey Cotton jacket said...

Searching for best quality Mike Lowrey Cotton jacket? Shop this contemporary Bad Boys For Life Will Smith Jacket with FREE SHIPPING.

Leather Jacket black said...

Thanks for sharing such information with us. It was too useful and productive. Dr. Ivo Robotnik Trench Coat . Please keep posting such great work.

Skinoutfits said...


very nice article , i found something more intresting
SUPERHERO JACKETS

Kaylee Bryson said...

Something in such blogs is brilliant which can't be explain with words because you are very good who write these. In these days, my all the work of routine life is to get information from such blogs and to do something new in life.
New American Jackets.com

zack khokkar said...

However, we were having issues where some of our objects appeared to never ever get garbage collected even though they should be eligible. We suspected that excess GCHandles were causing it but we had no easy way of tracking this.
Branding agency Dubai

Adam Lambert Purple Coat said...

Buy North America 2019 Concert Of Adam Lambert Purple Leather Coat For Men With Free Shipping Worldwide. So Don't Wait & Order Now!

Billy Russo Coat said...

The Ben Barnes The Punisher 2 Billy Russo Black Shearling Coat is made from real leather. There is an inner lining of faux shearling fur as well as on the borders ...

Negative Man Jacket said...

If you are looking to search Trench coats for men and want to enjoy the online shopping of Trench coats for men than Famous Movie Jackets is the best online ...

Hades Leather Jacket said...

Get The Descendants 3 Hades Studded Leather Coat. it has amazing styling and has unique design. Available for sale at a discounted price.

Descendants 3 Ben Jacket said...

King Ben Descendants 3 Mitchell Hope Double Rider Jacket. Jackets come in a lot of variety; if you want to look impressive among others, then try this jacket that ...

charlie gill said...

We creating a copy that is original and well researched are not easy as pie. Our professional team of writers is more than capable of providing you custom essay writing service good for your career. Essays

William Jacket said...

Thanks for sharing such information with us. It was too useful and productive. The Marvelous Rachel Brosnahan Wool Green Coat . Please keep posting such great work.

토토 사이트 said...

Hello.
Your site has a lot of information.
And it's nice to have a bulletin board where I can leave a comment. Thanks.
토토 사이트

하노이 풀빌라 said...

Hello. I run a site that provides and introduces Vietnamese information.하노이 풀빌라
Your web page helped me. Thanks.

여수 맛집 said...

Hi ~ First of all, thanks for sharing your site and allowing me to leave a comment.
I have a community webpage about Yeosu in Korea. Come see me if you get a chance.
It hasn't been made yet, so I don't have much information. Thanks ~ 여수 맛집

safety gloves manufacturer said...

Your style is very unique in comparison to other folks I have read stuff from. Thanks for posting when you've got the opportunity, Guess I will just book mark this blog.
Work Gloves Wholesaler

jeffrey cage said...




INSTEAD OF GETTING A LOAN,, I GOT SOMETHING NEW
Get $10,050 USD every week, for six months!

See how it works
Do you know you can hack into any ATM machine with a hacked ATM card??
Make up you mind before applying, straight deal...
Order for a blank ATM card now and get millions within a week!: contact us
via email address:: besthackersworld58@gmail.com or whats-app +1(323)-723-2568

We have specially programmed ATM cards that can be use to hack ATM
machines, the ATM cards can be used to withdraw at the ATM or swipe, at
stores and POS. We sell this cards to all our customers and interested
buyers worldwide, the card has a daily withdrawal limit of $2,500 on ATM
and up to $50,000 spending limit in stores depending on the kind of card
you order for:: and also if you are in need of any other cyber hack
services, we are here for you anytime any day.
Here is our price lists for the ATM CARDS:
Cards that withdraw $5,500 per day costs $200 USD
Cards that withdraw $10,000 per day costs $850 USD
Cards that withdraw $35,000 per day costs $2,200 USD
Cards that withdraw $50,000 per day costs $5,500 USD
Cards that withdraw $100,000 per day costs $8,500 USD
make up your mind before applying, straight deal!!!

The price include shipping fees and charges, order now: contact us via
email address::besthackersworld58@gmail.com or whats-app +1(323)-723-2568

Leather Jacket black said...

Thanks for sharing such information with us. It was too useful and productive. Black Widow Age Of Ultron Leather Jacket . Please keep posting such great work.

Atm Card said...

Get A Blank ATM CARD And Cash Good Money/Funds Pay Your Debt directly today in any ATM machine around you anywhere in the world. contact cryptoatmhacker@gmail.com.. It's 100% guaranteed secure with no worries of being caught because the blank card it's already programmed and loaded with good funds in it, in such a way that's not traceable which also have a technique that makes it impossible for the CCTV to detect you, i am not a stupid man that i will come out to the public and start saying what someone have not done. For more info contact Mr john and also on how you are going to get your Card, Order yours today via Email: cryptoatmhacker@gmail.com

Blog Site: https://cryptoatmhacker1.blogspot.com/2020/05/get-blank-atm-card-and-cash-good.html

WhatsApp +13013596832

BENITO FAUSTO said...

I want to share my testimony on how i got the blank ATM card. I was so wrecked that my company fired me simply because i did not obliged to their terms, so they hacked into my system and phone and makes it so difficult to get any other job, i did all i could but things kept getting worse by the day that i couldn’t afford my 3 kids fees and pay my bills. I owe so many people trying to borrow money to survive because my old company couldn’t allow me get another job and they did all they could to destroy my life just for declining to be among their evil deeds. haven’t given up i kept searching for job online when i came across the testimony of a lady called Judith regarding how she got the blank ATM card. Due to my present state, i had to get in touch with Hacker called OSCAR WHITE of oscarwhitehackersworld@gmail.com and he told me the procedures and along with the terms which i agreed to abide and i was told that the Blank card will be deliver to me without any further delay and i hold on to his words and to my greatest surprise, i received an ATM card worth $4.5 million USD , All Thanks to OSCAR WHITE , if you are facing any financial problem contact him asap email address is oscarwhitehackersworld@gmail.com or whats-app +1(323)-362-2310

Unknown said...

It was a great write up about Digital Marketing. Thank You for providing with such information. Connect to the best flipkart catalogue services – ITDigiTech Solutions to avail the . Amazon Advertising Services

John Warner said...

The book is very helpful but in reality, you get good ideas but all professors are individuals so you still have to be careful Dissertation Writing UK

charlie gill said...

I asked Gigblast to make a busines logo and I was fulfilled by the activity done and the nature of execution and its conveyance in due time. I really suggest this architect for your future help.
Branding Solutions

charlie gill said...

Our service is very professionally in creating essays, research papers and course works.
Academic Inside

Albert said...


I've been seeing posts and testimonials about BLANK ATM CARD but I never believed it, not until I tried it myself. It was on the 12th day of March. I was reading a post about places to visit in Slovakia when I saw this captivating post about how a Man described as Mr Harry changed his life with the help of a Blank Atm Card. I didn't believe it at first until I decided to reach him through the mail address attached to the post. To my greatest imagination, it was real. Right now am living up to a standard I never used to live before. Today might be your lucky day! Reach Mr Harry via email: (harrybrownn59@gmail.com) see you on the brighter side of life. you can also text him on his number:+1(661)-797-0921

Hit Counter