BP101T Unit 2 – Data Manipulation: NumPy & Pandas
Unit 2 covers scientific numerical computing with NumPy multidimensional arrays and structured tabular data management with Pandas. It explores vectorized batch scaling for pharmaceutical formulations, importing quality control data, handling missing values, filtering out-of-specification (OOS) batches, and computing key statistical metrics.
📋 Prescribed Syllabus Topics
📖 Detailed Lecture Notes & Solved Examples
11. Introduction to NumPy & ndarray Architecture
Dedicated Webpage➔NumPy (Numerical Python) is the core scientific computing package in Python. Standard Python lists store pointers to scattered objects across memory, requiring type-checking at every access. In contrast, NumPy introduces the `ndarray` (N-dimensional array), which stores homogeneous elements in contiguous blocks of computer memory. This architectural difference allows vectorized calculations — performing arithmetic operations simultaneously across millions of array elements compiled directly in C, eliminating slow Python `for` loops.Normal Python list me har element alag-alag memory me hota hai jisse bade calculations slow ho jate hain. NumPy ka ndarray pura data ek saath continuous memory block me rakhta hai. Isse lakho tablets ka batch calculation bina 'for loop' lagaye ek second me fast calculate ho jata hai jise Vectorization kehte hain.
▶Vectorized Formulation Scaling for 500,000 Tablets
Computing raw material requirements for an industrial batch of multi-ingredient Paracetamol tablets in a single vectorized statement.
import numpy as np
# Ingredients per single tablet in milligrams: [API, Binder (PVP K30), Disintegrant (SSG), Lubricant (Mg Stearate)]
single_tablet_mg = np.array([500.0, 25.0, 30.0, 5.0])
batch_size_tablets = 500000
# Vectorized operation: Multiply entire array and convert to kilograms (1 kg = 1,000,000 mg)
total_raw_material_kg = (single_tablet_mg * batch_size_tablets) / 1e6
ingredients = ["Paracetamol API", "PVP K30 Binder", "Sodium Starch Glycolate", "Magnesium Stearate"]
for ing, qty in zip(ingredients, total_raw_material_kg):
print(f"{ing}: {qty:.2f} kg required for batch")
- •Homogeneous Data: All elements in an `ndarray` must share the same data type (e.g. `np.float64`, `np.int32`).
- •Contiguous Memory: Direct buffer storage enables extremely high-performance mathematical throughput.
- •Broadcasting: NumPy automatically expands arrays with compatible shapes to perform arithmetic operations without copying data.
- •Array Creation: `np.array()`, `np.zeros()`, `np.ones()`, `np.arange()`, `np.linspace()`.
22. Array Indexing, Slicing & Matrix Operations for Formulation
Dedicated Webpage➔In pharmaceutical formulation development, dissolution profiles and stability testing generate 2D matrices (rows = time points, columns = sample batches). NumPy provides zero-indexed indexing and slicing (`[start:stop:step]`) to extract specific time points, replicate runs, or calculate column-wise means and standard deviations.Lab me alag-alag time points (5, 15, 30, 45, 60 min) par dissolution test ka data 2D Matrix me banta hai. NumPy slicing se hum specific batch ka result nikalte hain aur `axis=1` se har samay ka average drug release nikal lete hain.
▶Multi-Batch Dissolution Profile Matrix Analysis
Analyzing drug release percentages across 3 distinct production batches at 5, 15, 30, 45, and 60 minutes.
import numpy as np
# Rows = Time points (5, 15, 30, 45, 60 min), Columns = Batches (Batch A, Batch B, Batch C)
dissolution_matrix = np.array([
[28.4, 26.9, 29.1], # 5 min
[54.2, 52.8, 55.0], # 15 min
[78.9, 77.4, 80.2], # 30 min
[91.5, 89.8, 92.4], # 45 min
[98.7, 97.2, 99.1] # 60 min
])
# Mean dissolution across all batches at each time point (axis=1 calculates row-wise)
mean_profile = np.mean(dissolution_matrix, axis=1)
std_profile = np.std(dissolution_matrix, axis=1)
time_points = [5, 15, 30, 45, 60]
for t, mean, std in zip(time_points, mean_profile, std_profile):
print(f"Time {t:2d} min: Mean = {mean:.2f}% | SD = ±{std:.2f}%")
- •`axis=0` performs operations down columns (per batch/variable).
- •`axis=1` performs operations across rows (per time point/sample).
- •Matrix multiplication `np.dot(A, B)` or `@` operator is used in linear pharmacokinetic compartment modeling.
33. Pandas Series & DataFrames for Clinical & QC Laboratory Records
Dedicated Webpage➔Pandas is the leading Python data manipulation library. It introduces two fundamental structures: the `Series` (a 1-dimensional labeled array) and the `DataFrame` (a 2-dimensional labeled spreadsheet with heterogenous columns). Unlike NumPy arrays, Pandas DataFrames can hold mixed data types (text batch numbers, dates, float assay percentages, boolean release flags), making them ideal for pharmaceutical QC reports and clinical trials.Pandas Excel sheet ki tarah tables banata hai. Isme Series (1 column) aur DataFrame (poori table with rows & columns) hoti hai. Isme ek hi table me Batch Number (text), Date, Purity % (decimal) aur Pass/Fail status sab ek sath store karke OOS (Out-of-Specification) batches filter kiye jate hain.
▶Automated QC Laboratory Batch Screening Script
Building a DataFrame from production batch assays, calculating statistical limits, and flagging Out-of-Specification (OOS) batches.
import pandas as pd
qc_data = {
"Batch_ID": ["B2601", "B2602", "B2603", "B2604", "B2605", "B2606"],
"Drug_Name": ["Ciprofloxacin"] * 6,
"Assay_Percent": [98.4, 101.2, 88.5, 99.8, 102.1, 94.0], # IP Standard: 90.0% - 110.0%
"Hardness_kg": [6.2, 5.8, 4.1, 6.0, 6.5, 3.8], # Target: >= 4.5 kg
"Disintegration_min": [8.2, 9.1, 14.5, 8.0, 7.9, 18.2] # Target: <= 15 min
}
df = pd.DataFrame(qc_data)
# Boolean filtering for compliance
df["Assay_Pass"] = df["Assay_Percent"].between(90.0, 110.0)
df["Hardness_Pass"] = df["Hardness_kg"] >= 4.5
df["DT_Pass"] = df["Disintegration_min"] <= 15.0
df["Batch_Status"] = df["Assay_Pass"] & df["Hardness_Pass"] & df["DT_Pass"]
print("--- QC Batch Release Summary ---")
print(df[["Batch_ID", "Assay_Percent", "Hardness_kg", "Disintegration_min", "Batch_Status"]])
| Method / Attribute | NumPy ndarray | Pandas DataFrame | Pharma Use Case |
|---|---|---|---|
| Data Homogeneity | Strictly homogeneous | Heterogeneous (mixed types) | Storing batch codes (str) + assays (float) |
| Labeling | Integer indices only | Custom column names & row index | Column headers for Assay, Hardness, DT |
| File Import | `np.loadtxt()` (numeric only) | `pd.read_csv()`, `pd.read_excel()` | Importing HPLC and dissolution data sheets |
| Missing Data | Limited NaN support | Built-in `.dropna()`, `.fillna()` | Handling missing lab readings in stability studies |
- •Pandas Series: 1D labeled array capable of holding any data type.
- •Pandas DataFrame: 2D table structure with labeled axes (`index` for rows, `columns` for column names).
- •Boolean Indexing: `df[df['Assay_Percent'] < 90.0]` isolates failing batches instantly.
- •`df.describe()` outputs summary statistics: count, mean, std, min, 25%, 50%, 75%, max.
44. Importing CSV/Excel Stability Data & Handling Missing Values
Dedicated Webpage➔Analytical instruments (HPLC, UV Spectrophotometers, Dissolution testers) export run logs in CSV or Excel format. Real-world laboratory datasets often contain missing entries due to instrument baselines, sensor disconnects, or sample loss. Pandas provides methods like `pd.read_csv()`, `.isna()`, `.dropna()`, and `.fillna()` to clean data before statistical analysis.HPLC aur UV machines se data CSV ya Excel file me nikalta hai. Agar stability testing me kisi mahine ka reading chhoot gaya ho (NaN), toh Pandas ke `.interpolate()` function se us missing value ka linear estimate nikal kar clean kiya jata hai.
▶Handling Missing Readings in Accelerated Stability Data
Cleaning a 6-month stability study dataset containing missing assay values.
import pandas as pd
import numpy as np
# Simulated stability dataset
stability_data = {
"Month": [0, 1, 2, 3, 6],
"Assay_Temp25C": [100.0, 99.4, 98.9, 98.2, 97.5],
"Assay_Temp40C": [100.0, 97.8, np.nan, 93.1, 89.4] # Missing Month 2 at 40°C
}
df_stab = pd.DataFrame(stability_data)
# Check for missing values
missing_count = df_stab.isna().sum()
# Impute missing value using linear interpolation
df_stab["Assay_Temp40C_Clean"] = df_stab["Assay_Temp40C"].interpolate(method="linear")
print("Cleaned Stability Data:")
print(df_stab)
- •`pd.read_csv('filename.csv')` and `pd.read_excel('filename.xlsx')` load external datasets.
- •`.isna()` or `.isnull()` returns boolean mask of missing cells.
- •`.dropna()` drops rows or columns containing missing values.
- •`.interpolate(method='linear')` estimates intermediate missing values along a continuous time series.
- Remember: In NumPy, `axis=0` operates along columns (downwards), while `axis=1` operates along rows (horizontally).
- Explain the difference between `pd.Series` (1D) and `pd.DataFrame` (2D) in 5-mark university answers.
- Understand why vectorized operations are superior to standard Python loops in large pharmaceutical datasets.