• Feeds

  • 微信架构的启示

    腾讯大讲堂中最近分享了周颢演讲的微信技术总监解读微信架构的秘密,看完视频的一些心得。

    技术微创新

    微信的技术设计上有很多微创新,看起来都很小,但是对于系统的稳定性、用户体验及开发敏捷都具有重要作用。

    前轻后重
    由于客户端升级不便,从技术设计上尽量利用后端的设计来减少依赖客户端升级的方法。如某个版本新增了群聊功能,按常规思路,需要所有客户端升级才能全部打通。微信采用服务器兼容的方法,在老客户端不升级情况下让其增加群聊的功能,通过在服务端将群聊协议转换成之前旧版兼容的协议返回给老的客户端。

    客户端辅助设计
    微信客户端做了很多非常规的功能,比如常规的客户端测速方法是登录阶段轮询测试多个IP来选择服务器,这样会带来流量及登录速度双方面的开销,因此微信选择的方法是服务端返回最佳的IP(可能是通过历史数据分析)。客户端另外实现了一些容灾能力的配合,当一个IDC访问出现异常自动选择另外一个IDC。

    流量控制
    由于大部分无线用户对流量非常敏感,为了防止由于客户端不可预知的bug如死循环导致“偷流量”,服务端增加用户流量实时分析的方法,可以在海量数据下找出流量异常的用户,并给这些用户强制下行终止连接信号。

    基础研发的策略

    从视频介绍来看,微信的基础研发与业务结合非常紧密,提到的有基础组件比如Client/Server框架,从原理上看类似Twitter Finagle,框架封装了大容量网络通讯及远程调用所需各种功能。此外介绍的一些监控与统计框架也较为先进,可以随时增加监控指标。

    大量可复用的基础组件让业务开发非常敏捷,周颢总结的基础研发策略是“实现已有经验的固化”。这和其他一些公司中的基础研发团队思路有所区别。业界中基础研发脱离生产闭门造车的情况并不罕见,一方面业务部门重复低层次开发现象严重,另外一方面基础研发对业务理解欠缺,开发的组件模块与业务结合不紧密,无法被有效利用。因此类似微信这样增强业务团队的力量,在开发业务同时总结抽象更多的基础组件或许是一种更为实用的发展思路。

    腾讯海量课程

    视频中多次提到腾讯内部的一种海量模型的培训课程,其中的经验或设计模式包括
    大系统小做
    将一个复杂的大系统分解成多个独立的、小的、简单的任务去完成,“分而治之”。

    柔性可用
    服务端系统通常不是0与1之间的选择,可以在极端情况下做一定优雅降级,在服务端代码中需要体现这些设计。

    Set模型
    类似最近讨论的cell架构,按一个服务按用户范围分成不同的小单元,每个小单元(cell/set)具备全部业务服务能力,当一个set发生故障时候,只会影响这一小部分用户。微信架构中所做的补充是,在set之间增加了容错处理,当一个set发生故障时,使用类似一致性哈希的算法,调用方可以自动切换到下一个set来存储,并且将新的位置记录在index上(类似GFS master),周颢自称是一个简化版的GFS。

    微信的协议

    里面提到了XMPP和类Sync的自定义协议,里面提到XMPP的缺点是流量大和消息不可靠。但是流量大并非XMPP主要矛盾,可以很简单将其映射成二进制协议。消息ACK也可以添加简单的扩展协议来实现。较繁琐的还是兼容CMWAP网关的设计。

    使用XMPP或者简化的XMPP标准协议有很多好处,类似的场景有业界广泛使用的open api基本都使用HTTP及JSON,并不是由于这两种协议优化或高效,而是其简洁并得到广泛的认知。一种标准协议的认知及扩展成本要比一个自定义协议小得多,XMPP流量大的问题可以通过转换协议来实现,比如用binary 1代表login等全部协商协议,2代表message,消息增量获取也可以通过自定义扩展协议来实现。标准协议可以让团队内部及新人的认知成本降低,每一个参与者都很容易想到代码及架构改进建议。而且微信目前也在构建开放平台,自定义协议在开放方面必然具备一些局限。

    其他

    分布式理论
    从微信的分享也看到,指导业界不同背景的团队(不管大小)的分布式理论主要还是Google及Amazon系列论文,对互联网技术的发展具有深远影响。微信借鉴了Quorum及Merkle tree的理论,创造了一种自定义的做法,用于实现一种分布式递增发生器。
    不过分布式递增算法其实有更简单的实现方法,twitter也有相关的公开博文,由于视频相关背景介绍比较简短,可能大家解决的需求具有差异,就不展开了。

    监控
    数千的监控项,可以在分钟发现系统的异常

    敏捷
    每天20个后台变更

    技术传承
    从视频和PPT来看,微信的技术体系是从头搭建的,可能有不少利用qqmail时代的基础代码,但与深圳团队并不存在太大技术沿袭或者传承关系。从另外一个角度也看到微信技术团队的战斗力。

    新人力量
    一位毕业生的创意:按SET分布,全量数据从2G减到200K(具体的情况待了解)
    另外一个新特性,漂流瓶及摇一摇,据说也是2个月的本科毕业生一周完成,而且上线后运行稳定。

    存储

    上图中可扩展设计中字段配置表的做法感觉略显繁琐,目前大部分NoSQL产品的schema free方式可能更易于维护。

    产品驱动

    如图,其中“允许发布十分钟前的变更”这样做法通常在大型团队通常会引起广泛质疑,单纯学习这种形式并非正解,如何在互联网产品上实现敏捷值得产品和研发进一步思考。

    如想及时阅读Tim Yang的文章,可通过页面右上方扫码订阅最新更新。

    « | »

    43 Comments  »

    1. 已转发,多谢分享。

    2. 可惜视频不清晰

    3. FI

      好文章,感谢

    4. 不知所言

      不知所言

    5. Daly

      那些schema free的储存策略某程度上都要把key保存到数据中,也许对于海量数据而言空间利用率也许是个问题。微信只记录一个schema-id, 很省空间。自己定制的储存引擎虽然有做轮子的感觉,但可以很有针对性的优化嘛

    6. linchen

      hi,我是优酷土豆网员工,我们现在有一个功能,将新浪的微博拉回土豆网,我们是根据微博里面的视频url来来判断视频的来源网站,需要调用短连接转换为长链接的结果.不过这个接口性能太差了,经常超时.顺便问下,有没有其他方案,谢谢

    7. linchen

      顺便说一下我的邮箱[email protected]

    8. “NoSQL产品的schema free”, XMPP,protocol Buffer, Thrift, 这些都是将“对数据对象的结构化描述”序列化后传送的一种方法,如采用我的CTreeNode技术,就实现CTreeNode数据表示各类数据(如同CObject一样),序列化CTreeNode数据本身,实现你在分布式空间编程如同在同一进程空间编程一样,轻便,高效,强大。远非XMPP,Thrift等可比。欢迎访问我的微博及demo
      http://weibo.com/smith2012

    9. Alan

      XMPP流量大的问题可以通过转换协议来实现,比如用binary 1代表login等全部协商协议,2代表message

      这个能否说的更具体一些

    10. 不错 ,谢谢分享。

    11. 不过分布式递增算法其实有更简单的实现方法,twitter也有相关的公开博文

      这个是否可以提供一下相关链接呢?

    12. fgajhpgjkjwgvxcuaemzepbdnlkuwwmsufbulyxzjvmhenrrczqvoachzwemfoyipy

    13. Thanx for sharing such useful post keep it up :)

    14. I’m really impressed with your article, such great & usefull knowledge you mentioned here

    15. Thank you for every other informative blog. Where else may I get that kind of info written in such a perfect approach? I’ve a project that I’m simply now working on, and I’ve been on the glance out for such info.

    16. chrisgail

      Hey there, You’ve done a great job. I will definitely digg it and personally suggest to my friends. I am confident they will be benefited from this website.
      http://www.affordable-dissertation.co.uk/dissertation-writing-services-uk/

    17. 
      من مطمئن هستم که درون طولانی اینترنت نبا
      هماننده بوسیله این مطلب حیات ندارد برای اینکه مطلب کامل و مستوفا ی پیرامون این مبحث است

    18. Thank you for sharing your thoughts. I truly appreciate your efforts and I am waitiong for your next post thank you once again.
      книга онлайн-казино frank-kasinos.com бесплатныеонлайн-игры для развлечения

    19. BH

      Hello there, You have done an excellent job. I will definitely digg it and personally recommend to
      my friends. I am sure they’ll be benefited frlm this site.

      Also visit mmy homepage …

    20. Hi fantastic website! Does running a blog such as this take a lot of work?
      I’ve no understanding of coding but I was hoping to start
      my own blog in the near future. Anyway, should you have any recommendations
      or techniques for new blog owners please share. I know this is off subject nevertheless I simply needed
      to ask. Thank you!

    21. Very informative and good complex body part of articles, now that’s user friendly (:.

    22. Good site you’ve got here.. It’s difficult to find quality writing
      like yours these days. I honestly appreciate people like you!
      Take care!!

    23. Good post. I learn something new and challenging on sites I
      stumbleupon every day. It will always be helpful to
      read content from other writers and practice something from their websites.

    24. SLOT เว็บตรงรองรับการเล่นในทุกระบบทั้ง ios และ android เล่นกับเรา PGSLOT89 ปลอดภัย 100% เกมสล็อตที่รวดเร็ว ปลอดภัย ให้บริการตลอด 24 ชั่วโมง ฝาก – ถอน ด้วยระบบออโต้a href=” https://www.pgslot89.com“> pgslot89

    25. Paragraph writing is also a excitement, if you be acquainted with then you can write otherwise it is complicated to
      write.

    26. Engage in the unmatched thrill in the skies with Love Cloud in Las Vegas, Nevada!
      Elevate your romantic endeavors, anniversaries, weddings,
      or proposals to unprecedented heights aboard our opulent twin-engine Cessna.
      Captain Tony ensures a seamless flight while you enjoy intimate moments 5,280 feet above the dazzling Las
      Vegas skyline. Our cozy cabin is equipped with a plush bed, red satin sheets,
      and a “pillow for different positions,” setting the stage for
      memorable mile high club adventures. For just $995, experience 45 minutes of pure
      ecstasy, with longer sessions available for extended pleasure.
      Whether you’re a adventurous couple, seeking to
      spice up the flames of passion, or a group of bold friends, Love Cloud caters
      to your wildest fantasies. Join the select ranks of satisfied customers, from romantic
      newlyweds to seasoned swingers, who have experienced the thrill of close
      encounters in the clouds. Don’t miss your opportunity to soar to new heights of ecstasy with Love Cloud.
      Book your flight today and prepare for an unparalleled
      journey where the sky’s the limit!

    27. corrupt the whole shebang is cool, I apprise, people you intent not cry over repentance!
      The whole is sunny, as a result of you. Everything works, say thank you you.

      Admin, credit you. Thank you for the vast site.

      Appreciation you deeply much, I was waiting to buy, like never before!

      go for super, everything works distinguished,
      and who doesn’t like it, swallow yourself a goose, and affaire
      de coeur its brain!

    28. not

      acquire the whole kit is detached, I guide, people you transfer not regret!
      The entirety is critical, thank you. Everything works, say thank you you.
      Admin, thanks you. Thank you as a service to the vast site.

      Because of you decidedly much, I was waiting to take, like in no way before!

      buy wonderful, caboodle works spectacular, and who doesn’t like it, believe yourself a goose, and dote on its brain!

    29. come by the whole shebang is detached, I advise, people you transfer not regret!

      The whole kit is fine, tender thanks you. The whole kit works,
      say thank you you. Admin, as a consequence of you.

      Appreciation you on the tremendous site.
      Credit you deeply much, I was waiting to take, like in no way previously!

      steal wonderful, everything works horrendous, and who doesn’t
      like it, corrupt yourself a goose, and dote on its brain!

    30. buy everything is detached, I encourage, people
      you transfer not feel! Everything is sunny, tender thanks you.
      Everything works, blame you. Admin, thanks you. Thank
      you on the great site.
      Thank you very much, I was waiting to come by, like not in any degree
      in preference to!
      steal wonderful, everything works spectacular,
      and who doesn’t like it, corrupt yourself a goose, and love its percipience!

    31. come by the whole shebang is unflappable, I guide,
      people you intent not cry over repentance!
      The whole is sunny, as a result of you. The whole kit
      works, show one’s gratitude you. Admin, credit you.
      Thank you on the cyclopean site.
      Appreciation you very much, I was waiting to believe, like in no way previously!

      steal wonderful, everything works distinguished, and who doesn’t like it, buy yourself a goose, and affaire de coeur its brain!

    32. Cocaine comes from coca leaves found in South America.

      While split second used in well-known medicine, it’s now a banned substance rightful to its dangers.
      It’s hugely addictive, unrivalled to healthiness risks like brotherly love attacks, deranged disorders, and mean addiction.

    33. O mais importante é que o número na balança seja
      reduzido de forma saudável. https://Jodoku.com/@paulofelipepir

    34. ok

      corrupt the whole kit is cool, I advise, people you transfer not feel!

      The whole is sunny, thank you. The whole works, say thank you
      you. Admin, as a consequence of you. Appreciation you
      as a service to the cyclopean site.
      Appreciation you damned much, I was waiting to believe, like never
      in preference to!
      accept super, the whole shooting match works distinguished, and who doesn’t like it, buy yourself a goose, and love its perception!

    35. come by the whole shebang is unflappable, I advise, people you command not cry over repentance!
      The whole is fine, tender thanks you. The whole shebang works, show
      one’s gratitude you. Admin, as a consequence of you. Tender thanks you for the tremendous site.

      Thank you damned much, I was waiting to come by, like not in any
      degree before!
      steal wonderful, caboodle works distinguished, and who doesn’t
      like it, believe yourself a goose, and affaire de coeur its brain!

    36. air

      acquire the whole kit is cool, I guide, people you transfer not regret!
      Everything is critical, tender thanks you. The whole shebang works,
      show one’s gratitude you. Admin, as a consequence of you. Acknowledge gratitude you for the great site.

      Thank you deeply much, I was waiting to take, like never previously!

      buy super, all works spectacular, and who doesn’t like it, swallow
      yourself a goose, and love its perception!

    Leave a Comment