Python non-broadcastable output operand with shape (855,1) doesn't match the broadcast shape (855,5)

[Tim_]

Can someone please help me with a prediction model for stocks? See code, i do get an error when trying to visualise the data.
I started this week with Python, so mainy copying/learning from others and hardly real knowledge. Thanks for your help, would appreciate it.


from keras.models import Sequential
from keras.layers import LSTM, Dense, Dropout, Bidirectional
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from google.colab import files
data_to_load = files.upload()

# Load the stock dataset
df = pd.read_csv('cmg_daily_historical-data-01-15-2023.csv')

# Scale the data between 0 and 1
scaler = MinMaxScaler(feature_range=(0, 1))
df[['Open','High','Low','Last','Volume']] = scaler.fit_transform(df[['Open','High','Low','Last','Volume']])

# Create a dataset with the Last column
X = df[['Open','High','Low','Last','Volume']].values
y = df['Last'].values

# Split the data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Reshape the input data to be 3D [samples, timesteps, features]
X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))

# Define the LSTM model
model = Sequential()
model.add(Bidirectional(LSTM(50, return_sequences=True), input_shape=(X_train.shape[1], X_train.shape[2])))
model.add(Dropout(0.2))
model.add(Bidirectional(LSTM(50)))
model.add(Dropout(0.2))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')

# Train the model
model.fit(X_train, y_train, epochs=100, batch_size=32, validation_data=(X_test, y_test))
y_pred = model.predict(X_test)

# Convert the predictions back to the original scale
y_pred = scaler.inverse_transform(y_pred)
y_test = scaler.inverse_transform(y_test)

# Creating a line chart to visualize the predictions
plt.plot(df['Time'], y_test, label='Actual Close Price')
plt.plot(df['Time'], y_pred, label='Predicted Close Price')
plt.xlabel('Time')
plt.ylabel('Last')
plt.legend()
plt.show()

 

[Deleted member 2829]

1. Please post code in code tags (especially with Python which relies on indentation). Use the >_ button in the toolbar.
2. Always post errors in their context, so it's clear from which line they originate. In this case, the traceback showing the line number.
3. Have you already Googled this error and see if there's a solution that works for you ?

 

[Johna]

Use the >_ button in the toolbar.

It's better to use </>

 

[Deleted member 2829]

It's better to use </>

Indeed, my mistake 😳 Thanks for correcting.

 

[Tim_]

Hi, thanks for the reply. I will take notice of your remarks and will adjust my question (already tried Google, but couldn’t help me further. As a beginner reading/adjusting code is difficult, sorry)

 

[Tim_]

from keras.models import Sequential
from keras.layers import LSTM, Dense, Dropout, Bidirectional
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from google.colab import files
data_to_load = files.upload()

# Load the CMG dataset
df = pd.read_csv('cmg_daily_historical-data-01-15-2023.csv')

# Scale the data between 0 and 1
scaler = MinMaxScaler(feature_range=(0, 1))
df[['Open','High','Low','Last','Volume']] = scaler.fit_transform(df[['Open','High','Low','Last','Volume']])

# Create a dataset with the Last column
X = df[['Open','High','Low','Last','Volume']].values
y = df['Last'].values

# Split the data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Reshape the input data to be 3D [samples, timesteps, features]
X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))

# Define the LSTM model
model = Sequential()
model.add(Bidirectional(LSTM(50, return_sequences=True), input_shape=(X_train.shape[1], X_train.shape[2])))
model.add(Dropout(0.2))
model.add(Bidirectional(LSTM(50)))
model.add(Dropout(0.2))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')

# Train the model
model.fit(X_train, y_train, epochs=100, batch_size=32, validation_data=(X_test, y_test))

y_pred = model.predict(X_test)

# Convert the predictions back to the original scale
y_pred = scaler.inverse_transform(y_pred)
y_test = scaler.inverse_transform(y_test)

# Creating a line chart to visualize the predictions
plt.plot(df['Time'], y_test, label='Actual Close Price')
plt.plot(df['Time'], y_pred, label='Predicted Close Price')
plt.xlabel('Time')
plt.ylabel('Last')
plt.legend()
plt.show()

Errors:
ValueError Traceback (most recent call last)

<ipython-input-1-1d6f33943cdb> in <module>
45
46 # Convert the predictions back to the original scale
---> 47 y_pred = scaler.inverse_transform(y_pred)
48 y_test = scaler.inverse_transform(y_test)
49


/usr/local/lib/python3.8/dist-packages/sklearn/preprocessing/_data.py in inverse_transform(self, X)
527 )
528
--> 529 X -= self.min_
530 X /= self.scale_
531 return X

ValueError: non-broadcastable output operand with shape (855,1) doesn't match the broadcast shape (855,5)


Remarks:
I hope i did send enough and not too much info. Thanks for helping me out, i'll try to learn fast

 

[Tim_]

Anyone got a clue? Thanks, appreciate it..