• Feeds

  • Thrift and Protocol Buffers performance in Java

    I’ve used Thrift for some log client in our system. I’m going to use Protocol Buffers as the internal communication protocol between our XMPP servers. But I am hard to believe from the thrift and protocol buffers Python performance comparison, that that Protocol Buffers is 4-10 slower than Thrift. I’m going to do some similar tests on Java.

    The test is very similiar as the Python test. the .proto and .thrift file are copied from the above python test.

    The .thrift content:

    struct dns_record {
    1: string key,
    2: string value,
    3: string type = 'A',
    4: i32 ttl = 86400,
    5: string first,
    6: string last
    }
    
    typedef list<dns_record> biglist
    
    struct dns_response {
    1: biglist records
    }
    
    service PassiveDns {
    biglist search_question(1:string q);
    biglist search_answer(1:string q);
    }

    The .proto content

    package passive_dns;
    
    message DnsRecord {
    required string key = 1;
    required string value = 2;
    required string first = 3;
    required string last = 4;
    optional string type = 5 [default = "A"];
    optional int32  ttl = 6 [default = 86400];
    }
    
    message DnsResponse {
    repeated DnsRecord records = 1;
    }

    From the document, I learn that the optional and default values are one of the benefits of both serialization libraries. A record that matches the default value does not need to be included in the serialized output.

    I wrote up a simple test program to compare thrift, protocol buffers. I tested the serialize and deserialize together, because this is the most called part in most scenarioes.

    Test 1: 10,000,000 times

    ProtoBuf Loop  : 10,000,000
    Get object     : 15,130msec
    Serdes protobuf: 68,600msec
    Objs per second: 145,772
    Total bytes    : 829,996,683
    
    Thrift Loop    : 10,000,000
    Get object     : 12,651msec
    Serdes thrift  : 36,904msec
    Objs per second: 270,973
    Total bytes    : 1,130,000,000

    Test 2: 1,000,000 times

    ProtoBuf Loop  : 1,000,000
    Get object     : 1,094msec
    Serdes protobuf: 7,467msec
    Objs per second: 133,922
    Total bytes    : 83,000,419
    
    Thrift Loop    : 1,000,000
    Get object     : 524msec
    Serdes thrift  : 5,969msec
    Objs per second: 167,532
    Total bytes    : 113,000,000

    The serde_* functions are the times needed to serialize, and de-serialize the java object to and from a byte[].

    The result in Java was that Protocol Buffers 1.2-2 times slower than Thrift. (in the python test was 4~10 times). And PB binary size is smaller than Thrift. I think this is acceptable, and Google may improve the Protocol Buffers performance in the future version.

    Download my test code in Java: thrift-protocol-buffers-java.tgz,

    More information about thrift and protocol buffers: Thrift, Protocol Buffers installation and Java code howto

    Update: There is another Thrift vs. Protocol Buffers compare non-performance factors.

    UPDATE 2 (Apr 17): There is a performance tuning parameter optimize_for = SPEED (thanks Steve Chu) for Protocol Buffers, please see my next performance tests Thrift and Protocol Buffers performance in Java Round 2

    如想及时阅读Tim Yang的文章,可通过页面右上方扫码订阅最新更新。

    « | »

    19 Comments  »

    1. litaocheng

      .thrift 和 .proto content貌似是一样滴哦。呵呵
      受教了,2者性能相差无几。。

    2. I’ve said that least 4031754 times. The problem this like that is they are just too compilcated for the average bird, if you know what I mean

    3. I really love your writing, unlike most blogs I actually learn things, find the content useful and it is well written. 11/10 every time!

    4. This article is very good thanks to the owner of this blog is to be taken

    5. Thank you. please visit me about game should come with an illustrated guide with hundreds of pages that will come in handy even if you are in the not fun.

    6. Thanks for the work

    7. thank for this post good owner of this blog is to be taken

    8. raywill

      I think that pb is designed for C++/C mainly, we should always be aware of this

    9. What i don’t understood is if truth be told how you’re not actually a lot more well-preferred than you may be now. You’re very intelligent. You already know thus significantly on the subject of this subject, produced me in my view consider it from so many various angles. Its like men and women aren’t interested unless it is one thing to do with Woman gaga! Your own stuffs nice. All the time maintain it up!

    Leave a Comment