God's in his heaven.
All's right with the world.

0%

背景

在MySQL 5.1中,如果配置项sync_binlog=1,并且innodb_flush_log_at_trx_commit=1,那么MySQL的TPS将会下降到几十每秒,完全不可接受。这是因为InnoDB提交事务时,不仅需要将REDO刷盘,还需要将Binlog刷盘,每个事务都需要2次sync操作。机械磁盘的IOPS也就为几百的水平,所以InnoDB的性能极差。

这个问题,在MySQL 5.6中得到了比较好的解决。在了解Binlog Group Commit之前,需要先了解MySQL Binlog和InnoDB的两阶段提交。MySQL为了保证主库和从库的数据一致性,就必须保证Binlog和InnoDB的一致性,即如果一个事务写入了Binlog,InnoDB中就必须提交该事务;相反,如果一个事务没有写入Binlog,InnoDB就不能提交该事务。做法是:

阅读全文 »

在最近的一周时间里,一直都在研究和阅读 Amazon 的一篇论文 Dynamo: Amazon’s Highly Available Key-value Store,论文中描述了 Amazon 的高可用分布式键值存储服务 Dynamo 的实现原理。

dynamodb

之前在阅读 Google 的 Bigtable: A Distributed Storage System for Structured Data 时写了一篇 浅析 Bigtable 和 LevelDB 的实现 文章分析了 Bigtable 的单机版 LevelDB 的实现原理;在研究 Dynamo 时,作者发现 Dynamo 虽然和 Bigtable 同为 NoSQL,但是它们的实现却有着很大的不同,最主要的原因来自不同的应用场景和不同的目的。

阅读全文 »

在 2006 年的 OSDI 上,Google 发布了名为 Bigtable: A Distributed Storage System for Structured Data 的论文,其中描述了一个用于管理结构化数据的分布式存储系统 - Bigtable 的数据模型、接口以及实现等内容。

leveldb-logo

本文会先对 Bigtable 一文中描述的分布式存储系统进行简单的描述,然后对 Google 开源的 KV 存储数据库 LevelDB 进行分析;LevelDB 可以理解为单点的 Bigtable 的系统,虽然其中没有 Bigtable 中与 tablet 管理以及一些分布式相关的逻辑,不过我们可以通过对 LevelDB 源代码的阅读增加对 Bigtable 的理解。

阅读全文 »

LevelDB 介绍

LevelDB 是由 Google 开发的 key-value 非关系型数据库存储系统,是基于 LSM(Log-Structured-Merge Tree) 的典型实现,LSM 的原理是:当读写数据库时,首先纪录读写操作到 Op log 文件中,然后再操作内存数据库,当达到 checkpoint 时,则写入磁盘,同时删除相应的 Op log 文件,后续重新生成新的内存文件和 Op log 文件。

LevelDB 内部采用了内存缓存机制,也就是在写数据库时,首先会存储在内存中,内存的存储结构采用了 skip list 结构,待达到 checkpoint 时,才进行落盘操作,保证了数据库的高效运转。

LevelDB 总体架构

resources structure

阅读全文 »

Question

Can somebody explain me how a process can acquire SIX lock on a page? I my deadlock-graph xml file I see that a process running under RC isolation level (executing a select statement at the moment of deadlock) holds a SIX lock on a page.

What does this mean and how that lock could have been acquired? From what I got from http://msdn.microsoft.com/en-us/library/aa213039%28v=sql.80%29.aspx SIX locks protects S-locks on all resources and IX locks on some resources lower in the hierarchy.

For my case that would be IX-locks on rows? Can IX-lock be placed on a row? (I guess no). I am confused.

Another thing is that I expect several X-locks on rows and no S-locks at all (since the IL is ReadCommited). Why do I have the whole page locked with SIX if I only inserted several records in previous statement?

up!

阅读全文 »

The I/O from an instance of the Database Engine includes logical and physical writes. A logical write occurs when data is modified in a page in the buffer cache. A physical write occurs when the page is written from the buffer cache to disk.

When a page is modified in the buffer cache, it is not immediately written back to disk; instead, the page is marked as dirty. This means that a page can have more than one logical write made before it is physically written to disk. For each logical write, a transaction log record is inserted in the log cache that records the modification. The log records must be written to disk before the associated dirty page is removed from the buffer cache and written to disk. SQL Server uses a technique known as write-ahead logging that prevents writing a dirty page before the associated log record is written to disk. This is essential to the correct working of the recovery manager. For more information, see Write-Ahead Transaction Log.

阅读全文 »

The I/O from an instance of the SQL Server Database Engine includes logical and physical reads. A logical read occurs every time the Database Engine requests a page from the buffer cache. If the page is not currently in the buffer cache, a physical read first copies the page from disk into the cache.

The read requests generated by an instance of the Database Engine are controlled by the relational engine and optimized by the storage engine. The relational engine determines the most effective access method (such as a table scan, an index scan, or a keyed read); the access methods and buffer manager components of the storage engine determine the general pattern of reads to perform, and optimize the reads required to implement the access method. The thread executing the batch schedules the reads.

阅读全文 »

The following article applies to SQL Server versions 2008 +

Adequate memory is one of the most important factors for a well-functioning instance of SQL Server. By design SQL Server manages its own memory allocations via the SQLOS rather than having the servers Operating System perform this task.

Therefore it’s safe to say that monitoring SQL Servers memory use is a very important administrative task and in this post I am going to show you how to use Dynamic Management Views to take a closer look at how SQL is using memory and how these benefit troubleshooting activities.

Before we do that we need to see how much memory is on our server and how much is allocated to be used by SQL itself. This relates to the very first paragraph in this post, the difference between total memory and SQL memory is allocated to the operating system and how much that should be is really dependent on the total memory in the server. I have always started with a setting of 4Gb or 10% of the total memory, whichever is more and tested regularly.

阅读全文 »

Today I want to talk about another very interesting topic in SQL Server: disabling Row and Page Level Locks in SQL Server. Every time that you rebuild an Index in SQL Server, you can use the ALLOW_ROW_LOCKS and ALLOW_PAGE_LOCKS options to specify that SQL Server should acquire Row and Page Level Locks when accessing your data for reading and writing. Let’s look at what happens internally when we disable these locks.

阅读全文 »