[RFC] Improving Protocol Security via Timelocks

[eIP] Euler Improvement Proposals RFC
1

Title: [RFC] Improving Protocol Security via Timelocks
Author(s): @martin_shift, SHiFT Asset Management (https://shiftam.com/)
Submission Date: 6 February, 2023

Simple Summary
The proposal is to discuss the implementation of a timelock contract along with the existing 4/9 multisig (0x25Aa4a183800EcaB962d84ccC7ada58d4e126992) to safeguard protocol’s upgrades, including changes in individual asset configurations.

Abstract
It is imperative for a Euler Finance’s development to introduce a timelock on the upgradeable smart contracts, which will allow users to have some time to take necessary actions before a change is implemented in Euler Finance’s system of smart contract.

Motivation
With the growing adoption of DeFi among institutional investors it is necessary that DeFi protocols also provide a transparent mechanism to control smart contract exploit risks associated with providing liquidity to their platforms. Euler Finance is among most innovative lending protocols with a unique architecture, however the team multisig currently holds excessive power over deploying changes to the protocols smart contracts and, under certain conditions, might put user funds at risk. This RFC seeks to initiate a discussion on the issue.

Specification/Body Text
Despite being in early stage of development, DeFi has attracted a substantial number of institutional investors, who are more naturally more rigorous in their investment approach in comparison with retail investors. This means that DeFi platforms should strive to be more thorough in managing smart contract exploit risks and provide assurance to liquidity providers that their funds are safe.

EVM allows anyone to create open-source, immutable and censorship resistant smart contracts. Once deployed, an EVM smart contract cannot be changed, however, there is an option to make them upgradeable (e.g. using proxy contract). While it is desirable (and sometimes necessary) to have upgradeability in the protocol’s architecture, this flexibility comes with a caveat of additional attack vectors.

Any protocol could be drained by changing an implementation of a core smart contract with liquidity to allow a bad actor to withdraw funds. Another classic attack vector, particularly for lending markets, is changing risk parameters in a way that enables illiquid tokens with high centralization as collateral. This would allow for price manipulation via continuously (lending bad assets, borrowing stablecoins and buying that asset), which will eventually drain the market of “good” assets and result in bad debt for the protocol.

There is no question whether DeFi lending platforms (in their current form) need to have a degree of upgradeability to accommodate changes in risk profiles of specific assets available for lending/borrowing – it is vital to have the optionality of adding/delisting/updating risk parameters of assets to maximize efficiency of DeFi lending markets. However, these updates should be properly managed.

The two existing options for managing exploit risks are multisig and timelock smart contracts. The former option mitigates the risks of a single EOA having power over the protocol’s upgrades, while the latter allows LPs to have a specified time interval before changes are implemented to contracts. Multisigs have been proven to be less resilient to attacks, since signers could collude or compromise their access to multisig due to the human factor. Timelocks, in contrast, could be front run – suppose an exploitable bug was found in a protocol’s contract – by the time the team deploys the patched implementation of the contract there might be an exploit. So both options have their advantages and drawbacks. This is why a combination of these tools is used in top tier lending markets like Aave and Compound:

  1. Aave:
    Aave implemented a long (7 days) and short (1 day) timelock executor contracts in V2, their highest TVL lending market. The short timelock executor is used for risk parameter updates, while long time lock executor is used for governance and AAVE token upgrades.
    Additionally, Aave implemented the Guardian multisig with doxxed member who are voted on via governance proposals. This Aave Guardian serves as emergency backstop mechanism against hostile governance takeover, it can stop the implementation of a proposal. On top of that, Aave Guardian has the prerogative to pause Aave V2 market in case of emergency.

  2. Compound:
    As per documentation, each protocol’s contract is controlled by the Timelock contract, which can modify system parameters, logic, and contracts in a ‘time-delayed, opt-out’ upgrade pattern. The Timelock has a hard-coded minimum delay of 2 days. The Timelock contract queues and executes proposals that have passed a Governance vote. The Comptroller contract designates a Pause Guardian address capable of disabling protocol functionality. The Pause Guardian cannot unpause an action, nor can it ever prevent users from calling Redeem, or Repay Borrow to close positions and exit the protocol. COMP token-holders designate the Pause Guardian address, which is held by the Community Multi-Sig.

In the current Phase 1 the team multisig was granted the updateAdmin role, which can deploy new modules and update existing modules and set a new updateAdmin, and governorAdmin role, which can change risk configurations of assets on Euler’s market. Even though a Timelock contract exists, it can be bypassed by an attacker who has access to the multisig.

Euler Finance team declared their intention to transfer governorAdmin role from the team’s multisig to the Timelock contract deployment in Phase 2 and similarly transfer updateAdmin role in Phase 3, however no specific dates are mentioned by the team. Transition to Phase 2 assumes a decentralized and fully functional on-chain governance, which is the potential hurdle that is holding back the process.

It is imperative that Euler Finance adopts stronger security standards in order to facilitate the arrival of the institutional investors on Euler Finance. A possible solution could be prioritizing the migration of both governorAdmin and updateAdmin roles to a timelock contract on top of the existing multisig, while the governance takes its time to mature and become sufficiently decentralized. The multisig can be designed to have emergency pausing functionality where supply and borrow functions are banned, but users can still repay their debt and withdraw supplied assets. This approach is in line with the industry standards set by Aave and Compound. The community is welcomed to discuss any other options which can help the protocol mature and attract more institutional grade liquidity providers.

6 Likes
2

Thanks @martin_shift this is an important contribution and I hope people in the community will take the time to read this and provide feedback. I also want to highlight a recent proposal on Morpho for timelocks that people might find interesting as part of this discussion.

Let me take a moment to play Devil’s advocate on the idea of timelocks as a general security concept to help further the debate.

Are timelocks safer?
I definitely appreciate that timelocks help people using a protocol to make a decision on whether to stay or leave prior to a protocol code change, but I think it’s also worth emphasising that time locks bring their own substantial security issues, depending on how they are used. I do question the broad assumption that they are safer overall. Certainly the idea that there’s a ‘gold standard’ out there is something I would heavily dispute.

Compound has twice come a victim of latency in its governance processes. First, in the theft of COMP, where an additional $70m was stolen AFTER a bug was uncovered in its Comptroller due to latency in the governance processes prevented deployment of a realtively simple fix. Second, in the freezing of the ETH market on Compound, where $800m was unable to be withdrawn or liquidated for a week due to a bug in the cETH contract. Whilst no funds were lost in the latter example, they easily could have been had the market moved differently. Both these issues would have been resolved with limited losses in the absence of such a rigid governance procedure

On Aave, an issue with xSushi in 2021 led Justin Sun to withdraw $4.2B in a single day and caused chaos for a short time whilst governance debated what to do. And more recently, an attempt by the Mango hacker to squeeze people’s positions on Aave led to a rush to make wholesale changes in asset parameters across the protocol. Again, speed was in the essence in both cases, and whilst limit damage was done in the end, it easily could have been had market conditions worsened.

Do people need more time?
When a protocol code change comes, it is usually in one of two scenarios: a) slowly, after a prolonged discussion on governance forums; or b) quickly, because of a need for action due to some adverse conditions for the protocol.

In the case of a), one could argue that people already have the time they need to decide what to do before a change is implemented. Perhaps more clarity around exactly what code will be implemented and exactly when the code change update is executed could help here, but having the decision time locked wouldn’t necessarily give people more time to act.

It’s possible there’s also a Bystander effect with timelocks, where people have a false sense of security in the timelock period because they think everyone else is checking if something was implmented correctly or not. Empiricially, there’s some evidence to support this. The Compound bugs described above are good examples. Another comes from Moonwell (which also uses timelocks), where a mistake setting the collateral factor led to mass liquidations. This should have been very easy to spot if anyone was checking, but clearly people weren’t using the system as designed.

In the case of b), it’s questionable whether time locks are a good idea at all. Whilst good communication from everyone is crucial during such a period, having the tools to be able to respond quickly is crucial.

Anyway, these are just some contrarian thoughts designed to stir some debate here. Happy debating!

5 Likes
3

Commend Shift for this post. In strong agreement that this is critical work. Would also posit that delegate functionality is just as imporatnt to add as timelock and so including that in my comments below

@euler_mab I think your points are valid. Also appreciate the governance culture and open discussion you and team have cultivated arnd these things.

That said, imo the bigger point remains - that Euler as a project has become rather large, and so a multi-sig uninhibited by time with unidentified signers who aren’t delegated by governance is well below the security and safety standards we imo should be comfortable with.

I also think compound governor bravo is a sort of standard by now - maybe not “gold” - but given all the tradeoffs, imo it’s still best we have atm.

Overall, imo maturing governance contract from where it is today is urgent matter, and while tradeoffs would exist in any paths forward, I think most anywhere from current state would be progress.

Specifically, I can see at least these two paths forward to improve protocol governance / security

  1. implement timelocks into current multisig and add delegate functionality via something like www.metropolis.space as well

  2. transition Bravo contract governance

And again would like to learn more abt potential bespoke adjustments to each approach for solving time- sensitive matters. Would be neat if some core team members from Euler Labs had strong pov on where improvements could be made to existing solutions.

Overall I think it’s fair to say that upgrading the current system would be very valuable esp to support wider adoption, and so it should be priority matter. Particularly as next chapter of Euler growth seems so promising!

3 Likes
4

gm gm. Terrific question from @martin_shift and discussion.

One perspective to share from our coverage universe:
Though it came with some fireworks at the end, I think the system set up by Fei/Tribe DAO was really unique and could be a good case study. They had a lending protocol Rari, which was governed by the DAO with a timelock. During the hack, it needed an immediate pause, which was achieved through the “Security Gaurdian” that could pause the protocol or adjust some critical parameters. I am not sure of the implementation but if the DAO could veto the Gaurdian’s ability to pause it would be a good check on that Multisig.

Here is the post and discussion in their forum. Also some thoughts from Joey in a blog here.

Disregarding the optimistic governance structure (or not - it has its pros and cons), Fei had both a timelock and security gaurdian. This achieves both desires above around protection of users and also ability to react to bad news and protect users.

Hope helpful,
JTVL

2 Likes
5

Hello Euler community! My name is Ryan, and I currently work for Messari, covering Euler governance on a daily basis for Messari Governor.

I wanted to drop in as I noticed a great discussion taking place here, and I think this could lead to a valuable discussion for governance solutions ecosystem-wide.

Generally speaking, I think a deeper assessment of Euler’s governance would be necessary if we want to truly understand the benefits of adding timelocks. It could be considered irresponsible to implement timelocks without first understanding how they would be used for various situations.

My opinion is that governance decisions should not optimize for speed or disregard any security measures such as timelocks.
While there are multiple arguments to be made on both sides here, I think that proper implementation of timelocks is necessary in most cases, even if that means there are supplementary security considerations that need to be taken to mitigate the risk of the time delay.

As mentioned here, there has been extensive research on this topic by BGD Labs at Aave. Their process includes two timelocks for varying proposals, which works for them. What I don’t see mentioned here is why this is necessary and how this works for their specific case.

1380×924 129 KB

It is worth mentioning that this works well for Aave, but also notable that risk is mitigated via Certora’s continuous formal verification (Examples).

Also worth gaining a better understanding of the types of proposals that go through Aave governance. Since many proposals consist of timely parameter updates, there needs to be a way to execute proposals quickly.

As mentioned above, I believe that a careful assessment of Euler’s governance ambitions (what power should be obtainable via governance, and how much?) and future proposal cadence (types of proposals, frequency, and what risks are involved with these types of proposals?), and ultimate goals in governance should be thoroughly considered to better understand if an expedited design is necessary.

I think that @euler_mab brings up some interesting points on the overall effectiveness of the timelocks. I have a few thoughts to share:

While there are instances where expedited governance would have benefited protocols, I think the alternative (not having a timelock, or having one that can be bypassed) is much worse. The most obvious case for timelocks in governance is the Beanstalk exploit where a timelock would have halted the exploiter.

While I see the connection made to timelocks here, I disagree that timelocks are the right place to point fingers. The correct place to be looking here is the act of deploying contracts with bugs in the first place. Aave and Compound have seemingly come to similar conclusions, as they brought on Certora for Continuous Formal Verification to prevent potentially harmful upgrades (Aave <> Certora, Compound <> Certora). As mentioned above, implementing timelocks could result in the need for a supplementary solution like this.

In a perfect world, example A) is true. However, it may be naive to assume that exploiters are going to post a governance proposal and gather feedback prior to draining the DAO’s treasury. Regarding example B), the Aave short-timelock makes sense for this reason.

In conclusion, I think the timelocks would benefit Euler governance overall. While I likely can’t outline all the considerations for Euler’s risk analysis, I do think Euler should conduct deeper, more thorough research on the following:

  1. Euler Governance Aspirations.
  • What will the DAO vote on most in 3 months, 1 year, 5 years. Should there be some type of short-timelock (e.g., Aave)?
  1. Timelock Risks:

  • What roles of Euler governance could be negatively affected or burdened by the presence of timelocks?

    • Emergency Events: Funds are at risk due to an exploited bug or hack.
    • Collateral Parameter Changes: Risk of bad debt due to de-pegging or other major event requiring timely response/actions.
    • Efficiency: For routine, low-risk parameter changes, asset listings, etc.

  • How can the DAO design governance to safely alleviate these negative effects? (Some basic examples below…)

    • To alleviate the risk of funds being lost as a result of an exploited bug or hack, a DAO could implement guardian kill switches on rewards controllers, explore more elegant rewards contracts (optimistic rewards). In the case of borrow/lending .
    • Enable an emergency multisig to create the de-listing or lowering only of collateral assets in the event of adverse market conditions.
    • Implement tiered timelocks (i.e. Aave) to protect core assets/contracts within the DAO while allowing for efficient DAO operations on less critical decisions.

Hopefully this provides some helpful insight for the community :slight_smile:

6 Likes
6

thanks @raho - appreciate your points here. I think all makes sense. @John_TV_Locke rly like your example as well.

@euler_mab @martin_shift - wdyt?

Can we move forward by outlining a specific timelock solution that details benefits and risks to the DAO? Presumably Euler Labs would do this work, right? I like the framework that @raho proposed.

1 Like
7

gm

@euler_mab @martin_shift

Thoughts on this new governance framework from Element Finance? I’m not fully technical, but it seems this design may solve for all the issues and concerns we’ve articulated here. Very keen for everyone’s thoughts on this

https://messari.io/report/governor-note-evolving-on-chain-governance-with-element-council?__s=79c7m4aw555axsg2j0uh&utm_source=drip&utm_medium=email&utm_campaign=I'm%20gonna%20MakerDAO%20my%20money

2 Likes
8

Wow - nice to see this article in the wild! Happy to answer questions on the Element Council - it’s an impressive vision with some innovative use cases.

I will say that the framework is new and, therefore, relatively untested (compared to OZ/Compound Governor). That being said, it bridges the gap between some traditionally difficult governance trade-offs regarding on-chain governance. I’d be happy to connect Euler with the Element team if this is something Euler is considering!

2 Likes