1. What tricks does PCA do?
Simply put, PCA summarizes data by finding linear combinations of features. You can think of this as taking several photos of a 3D object, and then naturally sorting the photos from most representative to least representative before handing them over to the user.
If the input is the original data, PCA has two useful outputs. Z and W. By multiplying them, we can get the original data, the reconstructed data, but some information loss is acceptable (because we have reduced the dimensionality).
The following exercises demonstrate these two output matrices along with their data.
2. What you can do after applying PCA
After applying PCA to your data to reduce its dimensionality, you can use it for other machine learning tasks such as clustering, classification, and regression.
In the case of Taipei MRT later in this article, we will perform clustering on low-dimensional data. Here, some dimensions can be interpreted as the proportion of passengers at different times of the day, such as morning, afternoon, and evening. These stations are considered to belong to the same cluster since they have a similar proportion of daytime passengers (similar patterns!).
3. Let's take a look Our transportation dataset!
The data used here is hourly traffic data for the Taipei Metro Rapid Transit System, with the following columns: date, hour, origin, destination, passenger_count.
This example only keeps data for weekdays. This is because there are more interesting patterns between different stations on weekdays, such as stations located in residential areas having more commuters during the day, while stations located in business districts have more commuters in the evening. People come in.
The above plot is the hourly traffic trend (amount of passengers entering the station) for four different stations. The two red lines are Xinpu and Yong'an Market, which are actually in the overcrowded area of New Taipei City. On the other hand, the two blue lines are Taipei City Government and Zhongxiao Fuxing, where most companies are located and business activities are conducted.
This trend reflects the nature of both the area and the station, with the difference being most pronounced when comparing trends during commuting hours (7am to 9am and 5pm to 7pm). Masu.
4. Using PCA on hourly traffic data
Why reduce dimensionality before performing further machine learning tasks?
There are two main reasons.
- As the number of dimensions increasessince the distance between any two data points becomes closer, more similar and less meaningfulthis is called the “curse of dimensionality”.
- by higher dimension Due to the nature of traffic data, difficult to visualize and interpret.
Applying PCA allows you to identify the times when the traffic trends of different stations are most obvious and representative. Intuitively, from the plot shown earlier, we can infer that the times around 8:00 AM and 18:00 PM are representative enough to cluster the stations.
Remember that we talked about the useful output matrices Z and W of PCA in the previous section? Now we will interpret them in the case of MRT.
Original data, X
- Index: Stallion
- Column: time
- Value: Percentage of passengers admitted within a specific time (number of passengers / total number of passengers)
For such an X, we can apply PCA with the following code:
from sklearn.decomposition import PCAn_components = 3
pca = PCA(n_components=n_components)
X_tran = StandardScaler().fit_transform(X)
pca = PCA(n_components=n_components, whiten=True, random_state=0)
pca.fit(X_tran)
Specify the parameters here n_components This means that PCA extracts the three most important components.
Note that it is something like “”.If you take several photos of a 3D object, the photos will be sorted from most representative to least representative.” and select your top 3 photos.Therefore, if you set n_components At 5, you get two more photos, but the top three remain the same.
PCA output, W matrix
W can be thought of as the weight of each feature (i.e., time) related to the “photo”. More specifically, Main component.
pd.set_option('precision', 2)W = pca.components_
W_df = pd.DataFrame(W, columns=hour_mapper.keys(), index=[f'PC_{i}' for i in range(1, n_components+1)])
W_df.round(2).style.background_gradient(cmap='Blues')
Regarding the three principal components, we can see that PC_1 gives weight to the night time period, PC_2 gives weight to the noon time period, and PC_3 is closer to the morning time period.
PCA output, Z matrix
we can interpret Z A queue as an expression of a station.
Z = pca.fit_transform(X)# Name the PCs according to the insights on W matrix
Z_df = pd.DataFrame(Z, index=origin_mapper.keys(), columns=['Night', 'Noon', 'Morning'])
# Look at the stations we demonstrated earlier
Z_df = Z_df.loc[['Zhongxiao_Fuxing', 'Taipei_City_Hall', 'Xinpu', 'Yongan_Market'], :]
Z_df.style.background_gradient(cmap='Blues', axis=1)
In our case, we can assign names to the PCs because we interpret the W matrix and understand the potential meaning of each component.
The Z-matrix of these four stations shows that the first two stations have a large proportion of night time, while the other two stations have a large proportion of morning time. This distribution also affects the EDA findings (recall the line graph for these four stations from earlier).
5. Clustering PCA results using K-Means
After getting the PCA results, let's further cluster the transit stations according to their traffic patterns. This is represented by three principal components.
The Z matrix in the last section has station representations for night, noon, and morning.
We cluster stations based on these representations so that stations within the same group have similar passenger distributions in these three periods.
There are many clustering techniques such as K-Means, DBSCAN, and hierarchical clustering. Since the main topic here is to check the convenience of his PCA, we will skip the process of experimenting to see which technique is more suitable and use the following method: K-means.
from sklearn.cluster import KMeans# Fit Z matrix to K-Means model
kmeans = KMeans(n_clusters=3)
kmeans.fit(Z)
After fitting the K-Means model, let's visualize the clusters in a 3D scatter plot with plotly.
import plotly.express as pxcluster_df = pd.DataFrame(Z, columns=['PC1', 'PC2', 'PC3']).reset_index()
# Turn the labels from integers to strings,
# such that it can be treated as discrete numbers in the plot.
cluster_df['label'] = kmeans.labels_
cluster_df['label'] = cluster_df['label'].astype(str)
fig = px.scatter_3d(cluster_df, x='PC1', y='PC2', z='PC3',
color='label',
hover_data={"origin": (pca_df['index'])},
labels={
"PC1": "Night",
"PC2": "Noon",
"PC3": "Morning",
},
opacity=0.7,
size_max=1,
width = 800, height = 500
).update_layout(margin=dict(l=0, r=0, b=0, t=0)
).update_traces(marker_size = 5)
6. Taipei MRT transportation insights — clustering results
- cluster 0 :There are many passengers during the day, so it may be a “residential area” class.
- cluster 2 :In the evening, the number of passengers increases, so it may be a “business area” group.
- cluster 1 : There are many people who enter stations during the day and night, and it is more complex to explain the nature of these stations, as there may be different reasons for each station. Below we will look at his two extreme cases within this cluster.
for example, cluster 1, Taipei Main Station, the busiest station by number of passengers, is a huge transportation hub in Taipei, where commuters can transfer from the bus and rail systems to the MRT. Therefore, the pattern of heavy morning and evening traffic is clear.
On the other hand, Taipei Zoo Station is also in cluster 1, but it is not “full of people day and night''. On the contrary, there are very few residents living around this area, and most people rarely visit Taipei Zoo on weekdays, so there are not many people at either time of year.
The patterns of these two stations are not very similar, although they are in the same cluster. This means that cluster 1 may have too many stations that are not actually similar. Therefore, in the future, we will need to fine-tune the K-means hyperparameters, such as the number of clusters and techniques such as: silhouette score and elbow method It will be helpful.
