# Backtesting

Once stock features are generated, backtesting can be done using these pre-generated features. Backtesting is done independently per stock over a defined date range. For each day, the users are first filtered based on a predefined cutoff on each user feature.

for date in stock_features:

for user in stock_features[date]:

if (cutoff(user)):

continue

else:

user_weight = weight(user)

If the user passes the cutoff, their prediction is weighted based on the associated user features. Before weighting, these user features are transformed into a more usable form.

After transformation, all features are scaled appropriately so that feature values are standardized between all stocks. Each feature is both standardized and normalized to a scalar between 0 and 1. After being standardized, each feature is weighted based on the bull or bear sentiment of the corresponding tweet. If

*wa*and*wn*are the determined weights for the associated sentiment and non-associated sentiment feature values, the final feature value is$x = w_a*feature_a + w_n*feature_n$

Associated sentiment is defined as the the feature value (bull or bear) associated with the predicted sentiment and non-associated is the opposite feature value. For example, if a user tweeted

**bullish**about a stock and their return features were`return_bull = 10`

and `return_bear = 20`

, using the weights `w_a = 0.8`

and `w_n = 0.2`

, the resulting feature for user return would be `0.8*10 + 0.2*20 = 12`

. If the user had tweeted **bearish**, the result would be`0.8*20 + 0.2*10 = 18`

.After feature transformation, the user weighting function returns a normalized weight based on a given user prediction. This weight function is a linear combination of the user's features with their associated weight parameters.

$w_t = w_1x + w_2y + w_3z + ...$

After the user weight is computed, a corresponding time weight is calculated based on the tweeted/prediction time. This time weight is calculated by taking the time difference between 4pm on the current trading and the time of posting. This difference is then applied to a sigmoid function with the goal of giving more weight to tweets closer to 4pm.

Sigmoid function

def sigmoid(time_posted, curr_trading_day):

diff = time - curr_trading_day

x = diff + 5.2 # hours back where w = 0.5

return 1 / (1 + math.exp(-x))

The final tweet weight for each user is then defined as:

$w_{tweet} = w_{sigmoid} * w_{user}$

This final tweet weight for each user is first added to a bull and bear raw sum for each stock per day. The total raw weight per day is then computed by subtracting the raw bear sum from the raw bull sum based on pre-defined bull and bear weights.

$w_{AAPL} = w_{bull} * bull_{total} - w_{bear} * bear_{total}$

On some days, a stock may not receive many tweets, making a stock's total weight skewed by only a few users. A cutoff

*c*is added where any stock with less than*c*tweets will not be considered. Below is a time series example of the raw computed stock weights for ticker BYND each day for a pre-defined date range.Raw stock features ($BYND)

The computed stock weights for each day are stored along with their historical average and standard deviation. By using historical weights

*n*days before the current trading day, the stock weights are standardized. These weights are then mapped to their deviations from each stock's*n*days mean.Standardized stock features

Using each stock's daily weight, backtesting can be done alongside historical stock price data. First, on each trading day, a list of standardized stock weights are compiled. An example object below:

{

'2020-02-04': {

'AAPL': 1.23,

'TSLA': -2.1,

'F': 1.8,

...

},

'2020-02-05': {

...

},

...

}

A final cutoff

*c*is applied each trading day to all stocks by only considering stocks that satisfy the inequality,`abs(deviation) > c`

. Below are same standardized weights for $BYND but with the cutoff visualized.Standardized stock weights w/ cutoffs

Then, accuracy and theoretical returns can be calculated by comparing direction of stock deviation (positive/negative) with the stock's next day price movement from close to open.

Daily price % change ($BYND)

The red dots indicate the days the stock was chosen and the corresponding percent price change.

Last modified 3yr ago