The Ethereum merge, ever since Vitalik Buterin first touted it, has since been heralded with great promise of the possibilities that it would accrue to the Ethereum network: reduction in carbon footprint! Although first scheduled to happen in 2019, "the merge" did not occur not until September 15, 2022, 6:42 AM UTC+1. What is the merge, and why did it take such a while after it was announced to be finalized?
Ethereum is now using the Proof of Stake (PoS) consensus mechanism and previously used the Proof of Work consensus (PoW) mechanism. The PoW consensus relied on "Node Validators" who create blocks of transactions relying upon heavy mathematical algorithms where the Validator will calculate the block difficulty in order to validate a Block. The more computing power the node validator has, the better its chance of calculating a new block. This process requires a lot of computing power and, as a result, large energy consumption. PoS was then proposed as an alternative consensus mechanism.
PoS requires validators to stake ETH. The validators then check that new blocks proposed for the Ethereum network are not malicious. The Validator is also held to account by other network actors to ensure that no validator proposes the wrong block information. The staked ETH is used as collateral, such that when a validator authorizes/proposes a bad block, their staked ETH is then slashed. In the Proof of Stake mechanism, there is thus no need for heavy computation to propose new Blocks, thus leading directly to direct energy consumption in the Ethereum network.
The Ethereum network was not "switched off, then updated" for the merge. A famous analogy used to describe the merge event is that of a racing car whose engine was switched from a diesel engine to an electrical engine while the car is being driven on a highway; This underscores the level of the technical difficulty of the merge event. To achieve this, the newly proposed PoS mechanisms had to undergo critical testing alongside the PoW mechanism.
The Ethereum engineers proposed a road map that started with introducing a separate Ethereum chain called the "Beacon Chain" in December 2020. The PoS model was then used on the Beacon chain. The Ethereum Mainnet was still running on the PoW model at the time, and both chains ran parallel. This was so that errors that could happen due to the switch to the PoS model were quickly discovered. In the course of running the Beacon chain, "Shadow forks" were also implemented. Shadow forks were running PoS, and when bugs were found, they were immediately dropped.
The extensive testing continued, and in the course of the time, Testnets were successfully migrated to the PoS without any resultant errors. The significant change occurred when the Beacon chain was merged into the mainnet with all the state history and finalized at Block 15537394.