• Email:
  • Feeds

  • 单元化与分布式架构的切分问题

    单元化是将一个系统的架构按某种数据特征维度进行垂直的划分,比如网站有100万用户,如按照用户维度进行划分,则可以分成10个单元,每个单元存储10万用户资料。单元化的一些收益如下

    • 由于每个单元数据规模可控,相关维度内的所有资料可放在一个数据库中(如上例中的用户资料),不需要复杂的sharding分库分表逻辑,存储及缓存访问得到极大的简化。同时开发也变得简单,工程师不需要有丰富的“大规模大并发系统”开发经验。
    • 同时由于计算离存储更近,也可以让数据离用户更近,比如用户数据存储在地理上靠近用户的位置,数据有了更好的局部性(locality),因此也会获得更好的访问性能。部署上相关单元的前端、缓存、数据库、数据挖掘等节点可在同一个机柜,架构上让大数据的访问变得低廉,也在部分程度上让大数据更为快速及敏捷。
    • 可以自然支持不同用户分片支持不同的功能特性,天然的A/B testing试验场。

    分布式是将一个系统的数据分布到多个单元,以便使系统能够scale out,具有更好的可扩展性。当今大型网站基本上是分布式设计的。分布式系统除了机房内的,考虑到系统扩展性、用户访问的便捷性、机房规模的物理限制、异地容灾(比如2013年4月及8月的微信机房故障)等需要,大型系统也会考虑地理分布在多个机房。

    但是在社交网络中,由于数据的网状访问,单元化会碰到较难选择合适的单元化切分维度的问题,比如按用户或按内容进行单元化不能很好的适应数据访问局部性的问题,同时地理分布式也面临相似问题,由于社交网络中用户的页面需要访问的,所有地理分布的机房都同步需要全量数据,导致部署和维护成本较高。

    假定系统中存在一个跨单元的数据访问总线,并且总线的访问满足

    1. 局部性,单元内的访问,大部分的数据可以在单元内命中。
    2. 封装性,单元内的应用程序使用统一的方法访问数据,不需要关注数据的具体位置。

    那么这个数据总线应该如何设计?放在哪个层级比较合适?比如service layer, cache or storage(e.g. Google Spanner)?

    16 Comments  »

    1. arbow says:

      刚做完一个类似的项目,国外存放本地数据,国内存放全量数据,datachannel(数据访问总线)在DAO层,提供数据同步的接口代理以及MQ消息的传输

    2. Cycling Wear says:

      He is also a member of the American College of Healthcare Executives The American College of Healthcare Executives (ACHE) is an international professional association of healthcare executives (high-level hospital administrators, CEOs, COOs,Cycling Jerseys Men, health cycling shoes officers,Cycle Jerseys, etc. A couple of shots from the show bring out the animation style nicely and the rest of the cover is given over to the technical and production information. Believe me, they have done nothing

    3. Stun guns and tasers are not legal in some states. This airline covers most of the important cities of the bike shoes which include Ahmadabad, Mumbai,, New Delhi, Chennai, Bangalore, Goa, Kolkata,, Pune and many others. Why don’t we focus on the stuff that we are able to control. Alcohol is an especially powerful diuretic. Are you a stay at home mom with kids in school during the day? Iz Slovenske demokratske mladine (SDM) pa so sporocili, da so sokirani nad odlocitvijo vlade, da si “vsi dijaki

    4. Britain, for example, has reportedly been the home to a whole variety of big-cats for years maybe even for centuries. Every single girl says she wants to be unique. “There has been an adequate response,” Wellink said. “There is a story about the high heels. There are so much more somekeyword times of unintended hilarity to find out. Additionally, that aspirational customer who shops at Coach (COH), making $100,000,Parajumpers Gobi, who wants to live like he makes half a million, is completely un

    5. Low-cost erika kors totes for sale obtain a material damp

    6. I am sure this piece of writing has touched all the internet people, its really
      really nice paragraph on building up new weblog.

    7. Lost some sort of medicine personal good care will really currently have acute dangers since it can also be online business demonstration that could proceeds any present right this moment Then again it is typically dismaying to evaluate a lot of males and females ignore which will challenges a lot like program smell are actually really all of so that you can uncomplicated to deal with getting direct to the point possibilities the same as simply using a deodorant or maybe a perfume akin to Calvin

    8. Ebony says:

      It’s awesome to pay a quick visit this website and reading the views of all colleagues
      on the topic of this article, while I am also zealous of getting know-how.

    9. LinkPolonia often is the leading Online Polish Dating Agency that will help discover your soulmate in Polish community online. You will want to just sign up for free and choose Polish while your search option. Finding up to you from tons of singles in community of Polishes has now become easy.There are a lot stuff that puzzle a person, and you of things is keeping his woman happy. I have faith that every man will believe me should they understand learn how to maintain their woman happy, they mig

    10. Hello, its nice paragraph about media print, we all understand media is a
      fantastic source of information.

    11. Back on the Gamirasu, Metin previously had arrived. The itinerant singer, he made an important dodgy located executing located at wedding events and even functions. He / she sincerely been a saz, a challenging necked, bouzouki including musical instrument, and also sang mournful Anatolian land songs, tuhfeih1234 many of which looked like there was with regards to the assessments involving misplaced really like, Turkish design. The audience along at the Boat T was really a very young (this is exa

    12. This really is just the the reason why just a few often cling on the top! Personality: Personality and look work together specially when you are serving a major international market, you typically end up being within your best form. You simply cannot have the ability to be lax or complacent.How will you make it happen? Well, you cant grab their hands on their hands making them choose the Buy Now button. Nevertheless you are capable of doing specific things that will help them want to click on it

    13. Anonymous says:

      For example,Fitflop Flop Outlet Store, the phone call that you make to the prospect asking for the appointment is a key practice. Get sprinklers, water hoses or watering cans to irrigate your plants. Heavy Backpack SyndromeHeavy backpacks can be damaging to a child’s posture. It goes a bit deeper though as we learn that the previous coach,Fitflop Shoes Outlet, who is now working with just the Portable’s,Fitflop UK, did his best to try and get at least some of the se

    14. Amazing! This blog looks just like my old one!
      It’s on a entirely different subject but it has pretty
      much the same layout and design. Great choice of colors!

    15. What’s up mates, how is the whole thing, and what you want
      to say on the topic of this paragraph, in my view its
      genuinely awesome in support of me.

    RSS feed for comments on this post, TrackBack URI

    Leave a Comment