Monday, March 11, 2013

Breaking: The Blockchain has Forked

Developers are currently holding an emergency discussion in #bitcoin-dev to determine a way forward.

The gist of the problem is that 0.7 and 0.8 clients have diverged with each building a different blockchain because 0.7 clients did not accept 0.8 blocks from a certain point.

The problem was likely caused by large blocks being rejected by miners running the 0.7 client.

At this point mining pools, merchants, and exchanges are considering taking themselves offline until a way forward is agreed upon. As a result, you may see that your transactions are slow to confirm, though that might be a moot point if the blockchain is rolled back.

This is a dark day for Bitcoin. Implications for the exchange rate will likely be huge... should Bitcoins be sold off prior to the exchanges going offline.

Update: Mt.Gox has temporarily suspended all deposits.

Update 2: The exchange rate has dropped 10% in the last 10 minutes, now $44/Bitcoin.

Update 3: Gavin Andresen has sent out an emergency alert: "URGENT: chain fork, stop mining on version 0.8"

Update 4: It appears as though the solution is going to be to encourage miners to revert to mining on version 0.7 until THAT blockchain is longer than the 0.8 chain, which would then make it the master blockchain.

Update 5: As the miners have taken their machines offline, the rate of block confirmation has dropped substantially. Only one block has been confirmed in the last hour, and even then, the transaction volume on that block was only 25 Bitcoins. It appears that the mining pools are taking coordinated action to address the issue in a timely manner, as they switch over to Bitcoin v0.7.

0143 GMT: Update 6: The v0.7 blockchain is currently 225436 blocks in length, while the v0.8 blockchain is 225446 blocks in length. Until 0.7 is longer than 0.8, the problem will persist.

0148 GMT: Update 7: This was the original announcement of the problem.

0151 GMT: Update 8: From Pieter Wuille, a great explanation of the cause of the fork:

0.7 and older nodes use BDB for storing the blockchain databases. It seems this database has a limit on the size of the modification it can make atomically to the database. With the larger blocks of the past days, it seems to have triggered the limit. The result is that 0.7 (by default, it can be tweaked manually) will not accept "too large" blocks (we don't yet know what exactly causes it, but it is very likely caused by many transactions in the block). Specifically, block 
000000000000015c50b165fcdd33556f8b44800c5298943ac70b112df480c023 (height=225430) with >1700 transactions.

However. 0.8 (which uses a different database system) has no such limit, and happily accepts the block. As the majority of the hash power was on 0.8, the longest chain ended up using this block, which is not accepted by older nodes.

The solution is to (for now) go back to the old chain, which has block 00000000000001c108384350f74090433e7fcf79a606b8e797f065b130575932 at height 225430.

0219 GMT: Update 9: The exchange rate is really taking a hit now, down to $41.40 USD, or a 20% drop.

At this point, it's become an over-reaction. As soon as the v0.7-compatible blockchain is longer than the v0.8 blockchain, everything will be back to normal. This has actually been an excellent demonstration of how a decentralized development team and community can come together to resolve a major issue quickly, with minimal impact to the system.

0230 GMT: Update 10: Unfortunately it isn't simply a matter of the miners mining 10 blocks to catch up the 0.8 chain. There is still is considerable amount of hashing power working on the now defunct 0.8 chain and mining pool operators are just beginning the process of switching over to 0.7 (see the thread here). This will be a battle that will last well into the night, and possibly into tomorrow, depending on how quickly the mining power can be switched over. If it wasn't for the existence of pools, this would be like herding mosquitoes, let alone cats.

0249 GMT: Update 11: An important comment on how the blockchain works:

"For any block on the chain, there is only one path to the genesis block. Coming from the genesis block, however, there can be forks. One-block forks are created from time to time when two blocks are created just a few seconds apart. When that happens, generating nodes build onto whichever one of the blocks they received first. Whichever block ends up being included in the next block becomes part of the main chain because that chain is longer. More serious forks have occurred after fixing bugs that required backward-incompatible changes. Blocks in shorter chains (or invalid chains) are called "orphan blocks", and while they are stored, they are not used for anything. When a block becomes an orphan block, all of its valid transactions are re-added to the pool of queued transactions and will be included in another block. "

0300 GMT: Update 12: You can watch the ongoing battle, here, at blockchain.info. The orphaned blocks are part of the 0.7 chain which is trying to catch up to the longer, 0.8 main chain.

0313 GMT: Update 13: As transactions are propagated through the network, they're being added to both forks of the blockchain. One thing to realize, however, is that the block rewards that miners are receiving on the current main chain will be invalid, as the 0.7 chain, which will soon take over, will not recognize those coins.

The exchange rate has already made a miraculous recovery, back to 44.00 USD/Bitcoin.

0321 GMT: Update 14: Please read this update from Gavin Andresen, Bitcoin's lead developer.

0325 GMT: Update 15: Another explanation of what happened tonight:

35 comments:

  1. Does this mean I need to exit bitcoin v .8 and start up .7 then? Restart its synchronicity using .7? I'm 93% completed it says.

    ReplyDelete
  2. No, unless you are solo mining or running a pool, you do not need to do anything.

    ReplyDelete
  3. Before I read this post I was updating from the v.6 beta to v.8. Should I revert back to v.6 or stay with v.8?

    ReplyDelete
  4. When news first hit of the problem, I downgraded to 0.7 from 0.8. Now it seems I didn't have to do this, as I am not a solo miner. Bitcoin is syncing with the network, and my balance is now showing up as unconfirmed. Does anyone know if this will cause problems with my wallet? It seems like it won't, but I gotta admit I'm kinda nervous now...

    ReplyDelete
    Replies
    1. Your balance will be confirmed once you're synced up. Just hold tight and don't send any coins for the time being.

      Delete
  5. I think it's better to use 0.7x client.

    ReplyDelete
  6. You are safe if you upgraded to 0.8

    ReplyDelete
  7. The problem is not with the wallet, the problem is with the client 0.7 which was mining. It was not able to accept block chain height 225430.

    So,
    - if you are using 0.8 -- do not worry
    - if you are using 0.7 and not mining - do not worry
    - if you are using 0.7 and mining - stop mining and wait till proper instruction comes from the core development team.

    ReplyDelete
    Replies
    1. I think it's actually the other way around - if you are mining with the 0.7 everything is fine, but if you're using the 0.8 version, you need to stop mining and switch.

      Delete
    2. Switch to 0.7 if you are on 0.8.

      Delete
    3. No. If you're on 0.8 you should not do anything now. Upgrade to 0.8.1 when it's out.

      Delete
  8. Does anyone know if this forking issue is why Bitinstant.com cannot seem to transfer my cash to my BTC wallet? I know they got hacked but my cash has now been missing for two full weeks (and I'm not alone)

    ReplyDelete
  9. Guys -

    I literally just started learning and mining bitcoins yesterday and I'm a little fuzzy on what to do. I am using bitcoin-Qt 4.8.3 which is using v.0.8.0-beta. I am mining with GUIMiner (12-03-2012), connected to slush's pool. Should I stop anything? Uninstall something? I don't want to contribute to the problem and want to help if possible.

    Thanks
    Jon

    ReplyDelete
  10. you are mining on a pool that is addressing the issue. You are part of the solution Josh(not the problem) by helping hash up the correct block.

    ReplyDelete
  11. Thanks for the info, GoWest... I had a feeling I was worried over nothing

    ReplyDelete
  12. So if we are of slush's pool we help mining?

    ReplyDelete
    Replies
    1. slush just reported on IRC that he got his pool fixed, so he is now on the right fork.

      Delete
  13. Anonymous- I'm not certain if he has the problem solved yet but he is working on it which is better than some pools. Eligius and Eclipse Mining Consortium are the only 2 I am positive about, but if slush hasn't solved it yet he will shortly.

    ReplyDelete
  14. I see thanks Steve

    ReplyDelete
  15. Once this block no longer says "orphaned," all will be well... http://blockchain.info/block-index/357935/00000000000001c108384350f74090433e7fcf79a606b8e797f065b130575932

    ReplyDelete
  16. The wording "This is a dark day for Bitcoin." is a little strong, don't you think? It looks like very few coins were able to change hands before the price rebounded. Last trade was $44.50 USD on MtGox a couple minutes ago.

    ReplyDelete
    Replies
    1. At the moment it seemed appropriate. Turned out it wasn't a big deal.

      Delete
  17. Also remember that even if the community fail to de-fork the v0.8 chain and it stays as the main chain the only real world issues here seems to be a short-time possibility of double-spending that relies on luck and that will only last as long as v0.7 miners stay with v0.7.

    What the community is trying to do now is remarkable, but the real question for me is: If they instead ignored the fork and let it live, what would that mean for bitcoin as a whole?

    ReplyDelete
  18. Best Day for Bitcoin is more like it - another awesome proof of its survivability and stability, its back to normal on MtGox. BTC keeps appreciating in value while the USD is sliding into the crapper.

    ReplyDelete
  19. Thank you so much for keeping this blog updated to regularly, you've saved my heard from skipping a few beats, I've got all of my savings in BTC! Please provide your wallet address so I can send a donation, I'm sure I won't be the only one!

    ReplyDelete
    Replies
    1. Tip jar is on the left side of the blog (the QR code), thanks!

      Delete
  20. Could this be any more shaky?

    What happens when the govt trolls decide to hack it to death?

    Silver and Gold my friends.

    ReplyDelete
  21. Insanity. The day I put CASH into some invisible, untouchable internet venture is the day I insert my private parts into a hamburger grinder.

    Just Sayin'

    ReplyDelete
  22. the problem any sane person can see about bitcoin is - how the heck am i supposed to QUICKLY AND RELIABLY check if a given coin is not a counterfeit one.

    ReplyDelete
  23. Isn't BitCoin, just another Ponzi scheme? Why would anyone not buy silver and gold, instead.

    ReplyDelete
  24. The problem with gold is it can't be sent over borders and your government can confiscate it any time they want. And -- when was the last time you checked the purity of your bullion? A lot of people never even see the gold they buy, relying on electronic systems and trust to trade certificates.

    Bitcoin isn't anything like a ponzi scheme. A ponzi scheme is a scam that pays early investors interest out of incoming investment until new investors are left with nothing. Bitcoin is a currency; it doesn't pay interest; its value is determined by the market. Yes there's intense speculation in it right now and some investors might get burned, but you have the same thing with any commodity.

    If anything, Bitcoin is a bet on future government intrusion and control over the internet, and the collapse of fiat currency.

    ReplyDelete
    Replies
    1. Great reply! I couldn't have said it better myself. Viva la BTC!

      Delete