Introduction
AAVE is a pioneer in decentralized borrowing and a true blue chip DeFi protocol. As one of the most prominent borrowing protocols in the world, currently managing over 9 billion USD, robust risk management applications are required to empower users and developers with reliable, accurate data. The first step to reliable risk assessment is valid data. During our research for the AAVE v3 risk application, we observed inconsistencies between what was returned by AAVE subgraphs and results from directly querying contract on-chain. This post details our research process, proposed fixes and next steps.
Something is fishy 🐟
After chatting with the AAVE community, we opted to use the supported v3 subgraphs to fetch on-chain data. Utilizing the public subgraph increases the transparency and verifiability of the data.
We started implementing our queries and data normalization pipelines to build the required metrics. However, we quickly noticed that the data was incorrect. We started observing inconsistencies between data returned by the subgraph and what we were finding by directly querying the chain.
Let’s look at an example. We’ll look at the wallet 0xee2826453a4fd5afeb7ceffeef3ffa2320081268 on Avalanche.
We used the aave-utilities formatter to convert the graph output into a valid user summary. Take a look at the JSON here. As you can see, the WETH.e balance of the user is 1.2325.
{ "underlyingBalance": "1.232539847439213294", "underlyingBalanceUSD": "1830.71608619841228984408", }
But, the balance of this wallet on-chain is really close to 0.
This example highlights wrong wallet balances returned by the subgraph. Incorrect wallet balances caused various issues in our pipelines as we were calculating metrics such as value at risk, liquidation levels, wallets at risk, and more which are computed as wallet aggregations.
The Investigation 🧐
Subgraphs are meant to provide a powerful abstraction for developers to access in-depth data, without needing to dive deep into the internal protocol implementation. This abstraction breaks when the data is incorrect. Over a few days, we dived deep into the AAVE subgraph implementation + AAVE utils library. Ultimately, we couldn’t find any glaring issues with the AAVE codebase. We decided to follow a specific wallet and see where the balance accuracy started to break.
- We identified a wallet (0xbc1631afcb916bda28af42955fc970bf004596f8) on Polygon which withdrew all its supplied wETH.
- The user has no debt on chain.
- The subgraph shows that the user has a balance of aToken (supply balance).
At this point, our intuition was that the subgraph wasn’t handling events correctly. When searching the subgraph repo, we identified the events responsible for updating the balance fields:
- Burn
- Mint
Therefore, we focused on the events emitted during the Withdraw transaction.
Quickly, we noticed that the withdrawal amount was not equal to the value in the Burn. Instead, we observed that Burn.value + Burn.balanceIncrease = Withdraw.amount.
How could this be?
We dived deep into the implementation differences between v2 and v3. We discovered that the Burn and Mint events were changed in AAVE v3. In v3, The event value reports the amount that should be minted/burned while taking into account the interest which was accumulated (which impact the amount of mint/burned) since the last update.
In v2, the events reported the amount which was supplied/withdrawn, and the interest could be calculated using other fields in the subgraph like liquidityIndex, liquidityRate, etc.
Practically, it means that the value of the v3 events has changed to :
- Burn: value = (Withdraw amount) - (accumulated interest since the last update)
- Mint: value = (Mint amount) + (accumulated interest since last update)
Looking at the logic of burn handling in the subgraph, we see that only the Burn.value is used for calculations. This is inconsistent with the v3 updates and the root cause for the wrongly updated wallet balance.
Note that this issue affects all types of Mint and Burn events: AToken, VariableDebtToken, StableDebtToken.
The Fix
In order to keep consistency and backward compatibility to the structure of v2 (and to aave-utilities functions), The fix is pretty simple- we need to sum up the Burn.value and Burn.balanceIncrease and use this value for all the calculations. The opposite goes for Mint - we need to subtract Mint.balanceIncrease from Mint.value.
We submitted a PR in the AAVE subgraph repo which should fix the issue.
Significance for AAVE Community
Scope of Issue
- Any wallet that executed two actions with a delta such that interest was accrued stores the wrong balance.
- For example, a wallet that has performed the supply method twice or a combination of withdraw and supply.
- This bug also affects any field that is affected by interest accumulated by the wallet balances. Namely, this effects Reserve metrics such as totalSupply and totalBorrow per asset.
Impact
- This is a large-scale subgraph issue as it essentially affects all wallets that have ever interacted with v3 (barring wallets that have only ever supplied).
- This means that anyone consuming data on wallets/health/risk from the supported subgraphs was reading incorrect data.
- Any teams, traders, or institutions attempting to make data-driven decisions based on subgraph data are affected.
Short Term Mitigation
In the meantime, we've deployed subgraphs with our fixes. You can access them here:
What’s Next
With a fix for the AAVE v3 subgraph already deployed, v0 of the AAVE v3 Risk application is live! We’d love to get feedback and look forward to hearing from the community.
