Announcing the Winner of ‘User Behavior in DeFi Protocols’ Data Challenge
This challenge asked participants to gather their own data on their favorite DeFi protocol. From there, participants were asked to conduct exploratory data analysis, explore recommendations to the protocol, and dive into key metrics and user retention rates that correlate and precede the success of a given protocol.
In short, data scientists who competed in this challenge were asked to:
- 1. Create a dataset to conduct analysis.
- 2. Explain the methodology for dataset creation.
- 3. Analyze key customer metrics for all cohorts over time.
- 4. Build a report containing recommendations to the protocol to drive improvement in key business metrics.
The winners are as follows:
1st Place — Rodolfo Lima
Rodolfo is the crown winner of this data challenge and chose to conduct his report on the popular Uniswap Protocol.
The report and dataset can be retrieved for full analysis via the Ocean Market in the link below: https://market.oceanprotocol.com/asset/did:op:e58a57f2dc33fcc361079fe12d463dfd2e6cf7bea1d07a3d80fc5c9aa43b5f8d
The image above shows the retention trader of traders using Uniswap decentralized exchange. This chart follows the rise and decline of daily and monthly active traders on Ethereum Mainnet and correlates with the decline in market activity in the previous 12–18 months.
Interestingly enough, Rodolfo was able to highlight that volume and revenue accrued in the protocol have stayed lean through bearish trends. The image above details correlations between volume and revenue in the last 2 years (2021–2023).
A winning highlight in this report showed that a drop in average transaction value around June 2021 coincides with a significant drop in gas prices in parallel timeline. This indicates that when gas prices are lower, more users engage in transactions, even if they’re of a smaller value. When gas prices are high, only transactions that are of sufficiently high value (to justify the gas cost) are seen executed. Adversely, when gas prices are lower, a broader range of transaction values is recognized, including smaller ones.
There were 4 clusters of users that this report broke down to understand the behavior and tendencies of different users. The group clusters discovered are as follows:
- Cluster 0:
- Swap Count: Low (around 26 swaps on average)
- Volume in USD: Low (around $698,744 on average)
- Revenue in USD: Low (around $106,664 on average)
- Average Transaction Value: Low (around $15,980 on average)
- Average Gas Price: Moderate (around 62 on average)
2. Cluster 1:
- Swap Count: Very Low (around 1 swap on average)
- Volume in USD: Very Low (around $55,182 on average)
- Revenue in USD: Very Low (around $12,656 on average)
- Average Transaction Value: Moderate (around $27,547 on average)
- Average Gas Price: Extremely High (around 5,378 on average)
3. Cluster 2:
- Swap Count: Extremely High (around 54,127 swaps on average)
- Volume in USD: Extremely High (around $4.43 billion on average)
- Revenue in USD: Extremely High (around $558 million on average)
- Average Transaction Value: High (around $167,220 on average)
- Average Gas Price: Moderate (around 106 on average)
4. Cluster 3:
- Swap Count: Low (around 10 swaps on average)
- Volume in USD: Moderate (around $60.25 million on average)
- Revenue in USD: High (around $2.74 million on average)
- Average Transaction Value: Extremely High (around $7.69 million on average)
- Average Gas Price: Moderate (around 55 on average)
Recommendations Rodolfo gave to the protocol in the report we found to be efficient, thoughtful, and valuable. The report suggested a focus on the initial experience for new users and a simplified onboarding process. Optimized gas prices and offered alternatives like layer 2 integrations at the execution layer we thought would benefit any user of the protocol. Finally, a recommendation to not only understand where their users come from demographically, but to educate them on macroeconomic factors in markets, interest rates, reduced liquidity, slippage, and cycles was interpreted as a ripe suggestion to break down web3’s isolation from real-world economic dynamics.
2nd place — Saul Martin
Saul’s research and report can be found for reference inside the Ocean Market under these relevant links:
Flash loan statistics: https://market.oceanprotocol.com/asset/did:op:3997ba8b074842a20d4813ab8c17f61562b4a243d3ced4f0b548beac27ea74a5
User summary dataset: https://market.oceanprotocol.com/asset/did:op:1a0c57514f0089206b920924e5000071dda3341e9162f54f189df9890aa7a163
Full aggregated data: https://market.oceanprotocol.com/asset/did:op:f26948338b24cb17f9b71b1f09fa0a1f1763716658a9c3625bf3dcd55d0f93c7
Saul struck gold in a deep dive report into Aave, an Ethereum-based borrowing and lending protocol. Saul created a user data description cohort analysis that included a data frame of over 11,000 entries across 18 different user-specific tendencies and data points. The report later broke down 3 primary ways Aave has accrued revenue and stays afloat. 1) interest paid by borrowers 2) interest paid by flash loans and 3) liquidation fees.
The image above illustrates the comparative analysis of Total Value Locked (TVL) and borrowed amounts between AAVE v2 and the newly launched AAVE v3. AAVE v3 was introduced in March 2023 on Polygon.
Saul discovered an increase in the average size of flash loan amounts since the release of Aave v3 and determined that this is likely because of a decrease in premium fees per flash loan following the update.
The data, sourced by the Ethereum block explorer Etherscan, showed a steady decrease in users of the protocol retained over time. The data showed a 13% retention rate in new users after 2 months, and a 2.98% retention rate after 6 months. Liquidation rates are a leading indicator of users not returning to the platform. The report detailed that of the users who experienced a liquidation event, 60.8% did not engage in borrowing again.
Saul pinpointed 4 different problems in the protocol and recommendations for each, the results are as follows:
1. Strategically Review Flash Loan Fee Structures:
Problem: The introduction of AAVE v3 has indeed shifted user behavior towards its lower fees, but the discrepancy in fee structures between v2 and v3 is affecting revenue generation.
Recommendation: Considering a more unified and strategic fee model between versions may provide a balance between encouraging user engagement and maintaining revenue. Analyzing the specific needs and trends of 264 unique initiators and targets of flash loans could guide this adjustment.
2. Optimize Revenue Streams from Liquidations and Traditional Borrowing:
Problem: While flash loans contribute to revenue, the core remains traditional borrowing and liquidation fees. AAVE should consider fine-tuning its traditional borrowing rates to attract more users while maintaining profitability. Furthermore, since liquidations contribute to revenue but deter user engagement, an optimized liquidation process that minimizes user churn might be beneficial. This could involve implementing more user-friendly liquidation notifications, giving borrowers more time or options to rectify their positions before liquidation, or offering post-liquidation support and incentives to retain users.
Recommendation: Fine-tune borrowing rates and implement a user-friendly liquidation process to retain users. Increasing traditional borrowing and minimizing user churn from liquidations could lead to more stable revenue streams.
3. Strategize Borrowing and Lending Opportunities:
Problem: A significant portion of users (48%) are only engaging in depositing or withdrawing, with minimal borrowing activities, and there’s a notable difference in LTV ratios between stable and non-stable coins.
Recommendation: Develop tailored borrowing and lending products for various risk profiles, incentivizing both risk-averse and risk-seeking behaviors. For instance, provide different interest rate structures or rewards to promote borrowing of stable coins or create specialized products for high-risk traders. By encouraging more borrowing and lending activities, AAVE can increase its revenue from interest and transaction fees.
4. Enhance User Education and Risk Awareness:
Problem: Given that 60% of users facing liquidation do not engage in further borrowing, AAVE should invest in robust user education and risk awareness campaigns.
Recommendation: Tailored materials can guide users in understanding the potential risks associated with different assets and LTV ratios. Special attention should be given to educating the 12.2% of users trading in volatile assets at high LTVs, as they seem most vulnerable to liquidation.
3rd place — Narin Kunaseth
Narin’s full report and dataset can be retrieved for full analysis via the Ocean Market in the links below:
Uniswap User Dataset: https://market.oceanprotocol.com/asset/did:op:1d6b257ff29ec592f2efc1b1d3f151179c087e4a6f00ff59fdf66d4a0f375463
Full research report: https://market.oceanprotocol.com/asset/did:op:d61bb4a037da88d3239327d0a4df3c826f2a750c0242c0c5f148c4295fe2aaf5
Narin chose the Unicorn of DeFi to compose the report, Uniswap. Narin was awarded 2nd place for comprehensive research and insights on the ERC-20 swapping protocol from May 2020 to August 2023. This report gathered raw data about the Uniswap protocol from Flipside, which gave Uniswap v2 and v3 transaction data using blockchain explorers and APIs to retrieve it.
The image above details revenue made by the protocol since its inception. Each cohort of users (separated by their first transaction month). The revenue has been calculated using 0.3% of the total volume of swaps. As can be seen, the August 2020 cohort of users has generated nearly $800 million in revenue for Uniswap, which is significantly higher than other cohorts.
The review committee of data challenges through Ocean Protocol found this interesting to compare against statistics of active Ethereum wallets, daily/weekly/monthly transactions and specifically looking for indicators of successful protocols.
The next graph seen above shows a heat map of user retention by Uniswap over a 40-month period (May 2020 — August 2023). The takeaway from this report shows that Generally, new users tend to engage actively with the protocol during their initial months of participation, followed by a gradual decline in activity as time progresses. An exception to this trend is seen within the first six months, during which users sustain their interaction for nearly a year before experiencing a decrease.
Narin’s report displayed great quantitative work and data transformation to diagnose where and when shortcomings of the protocol took place. Most dates and declines shown in the graphics above fall on mutual timelines of industry-wide catastrophe events causing mass fear, uncertainty, and doubt in 2022 (FTX collapse, Terra LUNA, proceeding drama).
Finally, the report gave 3 recommendations for the protocol for a more sustainable and data-driven future:
- Simplify the onboarding process for new users.
- Create a campaign or community event tailored to the demands of older users of 2020–2021 to encourage continued use of the protocol.
- Divide users based on their total transaction volume to encourage loyalty and recurring customers.
Conclusion
Amidst a crypto winter, this challenge received a great turnout of participants interested in the subject matter of user behavior in DeFi protocols. This gave our core team a data-driven understanding of identifying best practices and leading metrics to determine the most quality and sustainable protocol practices in DeFi.
Are you a DeFi protocol that is looking to leverage a community of data scientists to offer intelligence and analytics to your protocol? If so, reach out to the Ocean Protocol Core team to discuss a future data challenge that highlights your protocol and questions you would like to have answered about your users, retention, and future best practices.
About Ocean Protocol
Ocean was founded to level the playing field for AI and data. Ocean tools enable people to privately & securely publish, exchange, and consume data.
Follow Ocean on Twitter or Telegram to keep up to date. Chat directly with the Ocean community on Discord. Or, track Ocean progress directly on GitHub.
