Understanding Cluster Analysis through Python Libraries

Discover how Python libraries simplify data clustering for better business insights…

Cluster analysis is a popular technique in data analysis and exploration that finds similarities between different groups of data, based on which datasets are classified or segmented into predefined clusters. It is an unsupervised machine learning algorithm and doesn’t need data that has been previously categorised or labelled. Instead, the algorithm identifies patterns and structures within the data on its own. Cluster analysis is widely used in various fields such as marketing, biology, finance, and social sciences for tasks like customer segmentation, anomaly detection, pattern recognition, etc.

Python, with its extensive libraries such as scikit-learn, SciPy, and PyClustering, provides a robust platform for implementing cluster analysis algorithms effortlessly. Its simplicity, versatility, and rich ecosystem make Python well-suited for conducting cluster analysis and interpreting complex datasets. Additionally, Python’s readability and ease of use contribute to its popularity in the machine learning community.

The versatility of cluster analysis makes it indispensable for uncovering hidden structures and relationships in data. This helps analysts to derive actionable insights and make data-driven decisions, as shown in Table 1.

Table 1: Use cases of cluster analysis

K-means clustering

Hierarchical clustering and K-means clustering are two popular techniques used for clustering data points into distinct groups. While K-means clustering divides data into a predefined number of clusters (which is represented by the letter ‘K’), hierarchical clustering creates a hierarchical tree-like structure to represent the relationships between the clusters. K-means clustering remains a popular method for cluster analysis, where the datasets are clustered into K clusters in such a way that the sum of the squared distances between the clusters and their assigned cluster mean is minimised. The Python algorithm iteratively assigns data points to the nearest cluster centroid and then recalculates the centroids based on the mean of the data points in each cluster. This process continues until convergence is achieved, resulting in well-defined clusters.

For instance, let’s say we have a dataset of 100 data points representing the annual income and spending score of customers in a mall. Using K-means clustering with K=3, the algorithm aims to partition these data points into three clusters in such a way that the sum of squared distances between each data point and its assigned cluster centroid is minimised. After several iterations, the algorithm converges, and we obtain three well-defined clusters. Customers in Cluster 1 might have moderate incomes and high spending scores, while those in Cluster 2 might have lower incomes and lower spending scores. Cluster 3 could represent customers with high incomes but moderate spending scores. This segmentation helps mall management understand different customer segments and tailor marketing strategies accordingly.

K-means algorithm has certain limitations as well. For instance, it might not always find the best solution, leading to less accurate clusters. Additionally, the algorithm requires the number of clusters (K) to be specified beforehand, which can be challenging in some cases. Moreover, K-means is sensitive to the initial placement of cluster centroids, and changing its starting points can lead to different final groupings. Despite these limitations, K-means remains a widely used and efficient algorithm for partitioning data into clusters, offering valuable insights into the underlying structure of datasets in various fields ranging from customer segmentation to image processing.

Minimum data and the number of clusters required

The minimum amount of data required for cluster analysis depends on various factors such as the complexity of the dataset, the number of features, and the desired level of granularity in the clustering results. If the number of data points is too small, the clusters may be unstable, and the clustering results may not be reliable. In general, one would typically need a dataset with at least a few dozen data points to capture meaningful patterns and relationships within the data to perform basic cluster analysis effectively. However, for more complex datasets or when dealing with high-dimensional data (like images with many pixels), hundreds or even thousands of data points may be required to obtain reliable clustering results. For instance, in image segmentation tasks where each pixel constitutes a data point, an image with 1000×1000 pixels would require at least a million data points for clustering to be meaningful.

The number of clusters chosen for analysis is also crucial. In market segmentation scenarios, where consumer behaviour is diverse, a maximum of 8-10 clusters may be appropriate to capture the different market segments effectively. Conversely, in crop classification tasks where the variety of crops is limited, a maximum of 4-6 clusters may suffice to categorise the crops accurately. Having enough data points per cluster is also essential to ensure stable and meaningful clustering. For instance, in financial risk assessment, each risk profile cluster should ideally contain a minimum of 100 data points to provide reliable insights.

In summary, while there is no fixed minimum threshold for the amount of data required for cluster analysis, it’s crucial to consider the dataset’s characteristics and the desired level of granularity to determine the appropriate number of data points and clusters accurately.

Clustering example based on synthetic data

Understanding customer preferences is crucial for businesses. In this section, we delve into the implementation of cluster analysis using Python to segment customers based on their coffee preferences. By generating synthetic data, we can create a controlled environment for showcasing the clustering process. Synthetic data allows for reproducibility and facilitates learning by providing a simplified yet realistic representation of the problem domain. Our goal is to showcase how businesses can utilise clustering techniques to gain insights into customer segmentation, enabling them to strategise marketing campaigns and product offerings effectively.

We use the NumPy library to generate synthetic data simulating customer preferences for two coffee types: cappuccino and filter coffee. This synthetic data allows us to demonstrate the clustering process without relying on real-world data, making it accessible for educational purposes.

# importing required python libraries
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans

We utilise np.random.seed (123) to ensure reproducibility in our experiments. By setting the seed value to 123, we ensure that the same random numbers are generated each time we run the code. This is crucial for consistency in our results, especially when working with random processes. n_samples = 1000 specifies the number of samples we will generate for our analysis, ensuring we have a sufficiently large dataset for meaningful insights.

# setting seed for reproducibility
np.random.seed(123)
n_samples = 1000

We generate random preferences for each customer by sampling from normal distributions. For male customers, we generate preferences for filter coffee with a mean of 0.61 and a standard deviation of 0.05, and preferences for cappuccino with a mean of 0.47 and a standard deviation of 0.05. For female customers, we generate preferences for filter coffee with a mean of 0.4 and a standard deviation of 0.05, and preferences for cappuccino with a mean of 0.65 and a standard deviation of 0.05. These preferences are randomly generated to simulate diverse customer preferences in our dataset.

# Generate random preferences for each customer
male_filter_coffee = np.random.normal(0.61, 0.05, size=(n_samples // 2,))
male_cappuccino = np.random.normal(0.47, 0.05, size=(n_samples // 2,))
female_cappuccino = np.random.normal(0.65, 0.05, size=(n_samples // 2,))
female_filter_coffee = np.random.normal(0.4, 0.05, size=(n_samples // 2,))

We combine the preferences of male and female customers for cappuccino and filter coffee into two separate arrays. By using the np.concatenate() function, we merge the arrays containing cappuccino preferences for males and females into a single array named cappuccino_preferences. Similarly, we merge the arrays containing filter coffee preferences for males and females into another array named filter_coffee_preferences. This consolidation allows us to analyse the combined preferences of both genders efficiently.

# Combine preferences of males and females
cappuccino_preferences = np.concatenate((male_cappuccino, female_cappuccino))
filter_coffee_preferences = np.concatenate((male_filter_coffee, female_filter_coffee))

We create a combined dataset called preferences, where each row represents a customer’s preference for cappuccino and filter coffee. By using the np.column_stack() function, we stack the arrays containing cappuccino and filter coffee preferences side by side, forming a 2D array. Additionally, we create an array called gender_labels to denote the gender of each customer. We assign ‘Male’ labels to the first half of the samples and ‘Female’ labels to the second half, ensuring that each set of preferences is associated with its corresponding gender. This combined dataset allows us to analyse and visualise the preferences of both male and female customers effectively.

# Combine preferences and gender labels
preferences = np.column_stack((cappuccino_preferences, filter_coffee_preferences))
gender_labels = np.array([‘Male’] * (n_samples // 2) + [‘Female’] * (n_samples // 2))

We perform K-means clustering to identify distinct clusters within the customer preference data. By specifying the number of clusters as 2 and setting a random state for reproducibility, we initialise the K-means object. Next, we fit the K-means model to the customer preferences data stored in the ‘preferences’ array. After fitting the model, we obtain the coordinates of the cluster centres using the ‘cluster_centers_’ attribute. Additionally, we obtain the cluster labels assigned to each data point using the ‘labels_’ attribute.

# Perform K-means clustering
kmeans = KMeans(n_clusters=2, random_state=42)
kmeans.fit(preferences)
cluster_centers = kmeans.cluster_centers_
cluster_labels = kmeans.labels_

We create a visualisation of the clusters using Matplotlib. By calling plt.figure(figsize=(10, 6)), we specify the size of the figure to be generated, ensuring it’s large enough to display the plot clearly. This helps us visualise the data effectively, making it easier to interpret the clusters.

# Visualize the clusters
plt.figure(figsize=(10, 6))

We plot the preferences of male and female customers on the same graph. For male preferences, we use blue markers, while for female preferences, we use red markers. By calling plt.scatter(), we create a scatter plot where the x-coordinate represents cappuccino preferences and the y-coordinate represents filter coffee preferences. This allows us to visualise the clustering of preferences for both genders simultaneously.

# Plot male preferences
plt.scatter(preferences[gender_labels == ‘Male’, 0], preferences[gender_labels == ‘Male’, 1], color=’blue’, label=’Male’)
# Plot female preferences
plt.scatter(preferences[gender_labels == ‘Female’, 0], preferences[gender_labels == ‘Female’, 1], color=’red’, label=’Female’)

We plot the cluster centres on the scatter plot to visualise their positions in the feature space. Using the ‘cluster_centers_’ attribute obtained from the K-means model, we extract the coordinates of the cluster centres. Then, we plot these points on the existing scatter plot using black ‘x’ markers to distinguish them from the customer preference data points. This helps us understand the centroids of each cluster and their relationship with the data points.

# Plot cluster centers
plt.scatter(cluster_centers[:, 0], cluster_centers[:, 1], marker=’x’, color=’black’, label=’Cluster Centers’)

We set the labels for the x-axis and y-axis of our plot using plt.xlabel() and plt.ylabel(), respectively. The x-axis is labelled as ‘Cappuccino preference’ and the y-axis as ‘Filter coffee preference’. Additionally, we set the title of our plot to ‘Clustering of male and female preferences for cappuccino and filter coffee’ using plt.title(). To distinguish between male and female preferences, we add a legend using plt.legend(), which assigns the colours blue and red to male and female data points, respectively. Finally, we display the plot using plt.show(), ensuring it includes a grid for better visualisation clarity.

# Set labels and title
plt.xlabel(‘Cappuccino Preference’)
plt.ylabel(‘Filter Coffee Preference’)
plt.title(‘Clustering of Male and Female Preferences for Cappuccino and Filter Coffee’)
# Add legend
plt.legend()
# Show plot
plt.grid(True)
plt.show()

With its simplicity, versatility, and rich ecosystem, Python continues to be a preferred choice for practitioners seeking to harness the power of cluster analysis for real-world applications. As organisations continue to grapple with the challenges posed by Big Data, Python’s role in cluster analysis is poised to grow even further. Its scalability, coupled with the availability of distributed computing frameworks such as Dask and Spark, equips practitioners with the tools needed to tackle large-scale clustering tasks efficiently. This scalability is particularly crucial in domains such as genomics and bioinformatics, where datasets can be massive and computational resources are often limited.

Furthermore, Python’s vibrant community and active development ensure that practitioners have access to the latest advancements and best practices in cluster analysis. Online forums, community-driven tutorials, and open source contributions foster collaboration and knowledge-sharing, enabling practitioners to stay abreast of emerging trends and techniques in the field.