Methodology of Accuracy Assessment
The accuracy assessment method involved evaluating the impact of signals provided by the AI within the subsequent 3-hour timeframe. For instance, if the AI Assistant recommended buying, the research team examined how many times the signal led to a higher price within the next 3 hours, observing at 10-minute intervals. This process allowed for the calculation of accuracy specific to that time interval.
To ensure a more reliable estimation, the assessment was conducted multiple times to reduce the impact of randomness. The research team repeated the evaluation process several times, gathering more data points, and then averaged all the obtained accuracies to calculate a more precise overall accuracy.
This assessment was carried out for all coins and across three levels of risk appetite. Additionally, the research team examined the signal resources most frequently suggested by the AI Assistant. It was observed that our system often received signals for a particular coin from the AI, possibly due to higher confidence in those signals. Alternatively, the AI Assistant may have prioritized signals from social resources for certain coins, as they showed a higher correlation with the movement of those specific coins.
The test period for this assessment was from 5th July to 7th July, during which the described methodology was applied to analyze the accuracy and signal resource preferences of the AI Assistant.
Overall Accuracy
Based on the research conducted in July 2023, the average accuracy of the AI Assistant among different user risk profiles for various coins was as follows:
- For Cautious users, the average accuracy of the Signals among
all coins was 63.66%.
- For Balanced users, the average accuracy of the Signals among
all coins was 63.16%.
- For Adventurous users, the average accuracy of the Signals
among all coins was 66.06%.
Detailed Accuracy Assessment
The detailed accuracy assessment for different coins and user risk profiles is as follows:
| Coin | Risk App | Accuracy | Attended Features |
|---|---|---|---|
| ADA | Cautious | 64.07% | Mostly attended Technical Indicators |
| ADA | Balanced | 59.93% | Mostly attended Technical Indicators |
| ADA | Adventurous | 60.18% | Mostly attended Technical Indicators |
| BNB | Cautious | 56.81% | Mostly attended Technical Indicators |
| BNB | Balanced | 55.14% | Mostly attended Technical Indicators |
| BNB | Adventurous | 59.03% | Mostly attended Technical Indicators |
| BTC | Cautious | 63.21% | Mostly attended Technical Indicators |
| BTC | Balanced | 65.23% | Mostly attended Technical Indicators |
| BTC | Adventurous | 64.11% | Mostly attended Technical Indicators |
| DOGE | Cautious | 65.21% | Mostly attended Technical Indicators |
| DOGE | Balanced | 61.79% | Mostly attended Technical Indicators |
| DOGE | Adventurous | 61.79% | Mostly attended Technical Indicators |
| DOT | Cautious | 78.67% | The coin behavior was based on social news |
| DOT | Balanced | 91.03% | Mostly attended On-chain features |
| DOT | Adventurous | 91.03% | The price predictions were well |
| ETH | Cautious | 60.88% | Mostly attended Technical Indicators |
| ETH | Balanced | 61.43% | Mostly attended Technical Indicators |
| ETH | Adventurous | 61.99% | The price predictions were well |
| LTC | Cautious | 66.43% | Mostly attended Technical Indicators |
| LTC | Balanced | 55.32% | The coin behavior was based on social news |
| LTC | Adventurous | 67.19% | The price predictions were well |
| SOL | Cautious | 77.75% | The price predictions were well |
| SOL | Balanced | 70.08% | Mostly attended Technical Indicators |
| SOL | Adventurous | 72.08% | Mostly attended Technical Indicators |
| XRP | Cautious | 48.04% | The coin behavior was based on social news |
| XRP | Balanced | 46.37% | The coin behavior was based on social news |
| XRP | Adventurous | 58.6% | The price predictions were well |
| TRX | Balanced | 65.32% | The price predictions were well |
| TRX | Adventurous | 64.65% | The coin behavior was based on social news |
| TRX | Cautious | 55.56% | The coin behavior was based on social news |