Good Data vs Bad Data: Real-Photo Version¶
Using two actual leaf photos, no simulation¶
This notebook uses two REAL photos directly:
- Good data = the daylight indoor photo (plain background)
- Bad data = the dim indoor photo (dark wood background)
Because the two photos differ in background, camera distance, and leaf angle, we cannot compare pixels directly like we could with a synthetic "bad" twin. So this notebook first aligns the two photos, warping the indoor one so the same physical point on the leaf lines up with the daylight photo, and only then trains and tests the model.
Run cells in order.
1. Upload your two real photos¶
Upload the DAYLIGHT (good) photo first, then the INDOOR (bad) photo.
from google.colab import files
import cv2, numpy as np, matplotlib.pyplot as plt
print("Upload the GOOD photo (daylight, plain background):")
up1 = files.upload(); DAY_PATH = list(up1.keys())[-1]
print("\nUpload the BAD photo (dim indoor, dark background):")
up2 = files.upload(); IND_PATH = list(up2.keys())[-1]
def show(im, title="", size=(4,5)):
plt.figure(figsize=size)
plt.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB) if im.ndim==3 else im, cmap='gray')
plt.title(title); plt.axis('off'); plt.show()
Upload the GOOD photo (daylight, plain background):
Saving WhatsApp Image 2026-07-25 at 14.27.49.jpeg to WhatsApp Image 2026-07-25 at 14.27.49.jpeg Upload the BAD photo (dim indoor, dark background):
Saving WhatsApp Image 2026-07-25 at 14.27.49 (1).jpeg to WhatsApp Image 2026-07-25 at 14.27.49 (1).jpeg
2. Load both photos and segment the leaf in each¶
GrabCut isolates the leaf independently in both photos, adapting to each background.
def load(path, longside=800):
im = cv2.imread(path)
h, w = im.shape[:2]
s = longside / max(h, w)
return cv2.resize(im, (int(w*s), int(h*s)))
def leaf_mask(im):
mask = np.zeros(im.shape[:2], np.uint8)
h, w = im.shape[:2]
rect = (int(w*0.08), int(h*0.08), int(w*0.84), int(h*0.84))
cv2.grabCut(im, mask, rect, np.zeros((1,65),np.float64),
np.zeros((1,65),np.float64), 5, cv2.GC_INIT_WITH_RECT)
m = np.where((mask==cv2.GC_FGD)|(mask==cv2.GC_PR_FGD), 255, 0).astype(np.uint8)
m = cv2.morphologyEx(m, cv2.MORPH_CLOSE, np.ones((15,15), np.uint8))
n, lb, st, _ = cv2.connectedComponentsWithStats(m)
if n > 1:
m = np.where(lb == 1+np.argmax(st[1:, cv2.CC_STAT_AREA]), 255, 0).astype(np.uint8)
return m
day = load(DAY_PATH)
ind = load(IND_PATH)
mday = leaf_mask(day)
mind = leaf_mask(ind)
show(day, "good data (daylight)"); show(ind, "bad data (dim indoor)")
print("daylight leaf area:", int((mday>0).sum()), "px")
print("indoor leaf area:", int((mind>0).sum()), "px")
daylight leaf area: 261434 px indoor leaf area: 192286 px
3. Align the two photos¶
Why this step is necessary. The two photos were shot separately, with different backgrounds, distances, and leaf angles. A direct pixel-by-pixel comparison would be meaningless without first lining up the same physical leaf point in both images.
Why shape-based alignment, not feature matching. The standard method (ORB keypoints + homography) failed here: the dim indoor photo has too little visual texture for reliable point matching, and it distorted the leaf's scale by nearly 2x. Instead we align using the leaf's own silhouette, its centroid, orientation and size, found through PCA on the mask. This only needs the segmentation from step 2, not fine texture detail, so it works even on a soft, low-contrast photo.
def leaf_pose(mask):
ys, xs = np.where(mask > 0)
pts = np.stack([xs, ys], 1).astype(np.float64)
mean = pts.mean(0)
cov = np.cov((pts - mean).T)
eigval, eigvec = np.linalg.eigh(cov)
order = np.argsort(eigval)[::-1]
eigval, eigvec = eigval[order], eigvec[:, order]
angle = np.degrees(np.arctan2(eigvec[1,0], eigvec[0,0]))
scale = np.sqrt(eigval[0])
return mean, angle, scale
m1, ang1, sc1 = leaf_pose(mday)
m2, ang2, sc2 = leaf_pose(mind)
best = None
for extra in [0, 180]: # PCA orientation is ambiguous by 180 degrees, try both
d_angle = ang1 - ang2 + extra
d_scale = sc1 / sc2
M = cv2.getRotationMatrix2D(tuple(m2), -d_angle, d_scale)
M[:, 2] += (m1 - m2)
warped_ind = cv2.warpAffine(ind, M, (day.shape[1], day.shape[0]))
warped_mask = cv2.warpAffine(mind, M, (day.shape[1], day.shape[0]))
iou = ((warped_mask>0)&(mday>0)).sum() / ((warped_mask>0)|(mday>0)).sum()
if best is None or iou > best[0]:
best = (iou, warped_ind, warped_mask)
iou, warped_ind, warped_mask = best
print(f"alignment quality (IoU, 1.0 = perfect): {iou:.3f}")
if iou < 0.7:
print("WARNING: alignment is weak. Results below may be unreliable.")
blend = cv2.addWeighted(day, 0.5, warped_ind, 0.5, 0)
show(blend, "alignment check: daylight + warped indoor overlaid")
alignment quality (IoU, 1.0 = perfect): 0.905
4. Build the ground-truth labels¶
Labels come ONLY from the daylight photo (the most reliable color record), using the LAB color system's a* channel: green below 0, transitioning 0 to 15, red above 15. We only keep leaf pixels where BOTH photos overlap after alignment, so every labeled pixel has a valid partner in both images.
leaf = (mday > 0) & (warped_mask > 0)
print("usable overlapping leaf pixels:", int(leaf.sum()))
lab = cv2.cvtColor(day, cv2.COLOR_BGR2LAB)
a = lab[:,:,1].astype(int) - 128
label = np.full(day.shape[:2], -1, np.int64)
label[leaf & (a < 0)] = 0
label[leaf & (a >= 0) & (a < 15)] = 1
label[leaf & (a >= 15)] = 2
for i, name in enumerate(['green', 'transitioning', 'red']):
print(f"{name:14s} {100*(label==i).sum()/leaf.sum():5.1f}% of leaf")
usable overlapping leaf pixels: 250291 green 3.5% of leaf transitioning 21.0% of leaf red 75.5% of leaf
5. Extract features from both REAL photos¶
Same six features as before, computed independently on the real daylight photo and the real (aligned) indoor photo.
def feats(im):
lab = cv2.cvtColor(im, cv2.COLOR_BGR2LAB).astype(np.float32)
hsv = cv2.cvtColor(im, cv2.COLOR_BGR2HSV).astype(np.float32)
g = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY).astype(np.float32)
tex = cv2.Laplacian(g, cv2.CV_32F)
return np.stack([lab[:,:,0], lab[:,:,2], hsv[:,:,0], hsv[:,:,1], hsv[:,:,2], tex], -1)
Fg = feats(day) # real daylight
Fb = feats(warped_ind) # real indoor, aligned
ys, xs = np.where(leaf)
np.random.seed(42)
idx = np.arange(len(xs)); np.random.shuffle(idx)
split = len(idx)//2
tr, te = idx[:split], idx[split:]
Y = label[ys, xs]
Xg = Fg[ys, xs]
Xb = Fb[ys, xs]
print("train pixels:", len(tr), "| test pixels:", len(te))
train pixels: 125145 | test pixels: 125146
6. Train and test three ways, using only real data¶
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import recall_score
def run(Xtr, Ytr, Xte, Yte):
clf = RandomForestClassifier(n_estimators=60, max_depth=14, n_jobs=-1, random_state=0)
clf.fit(Xtr, Ytr)
P = clf.predict(Xte)
rec = recall_score(Yte, P, average=None, labels=[0,1,2], zero_division=0)
macro = recall_score(Yte, P, average='macro', zero_division=0)
return dict(per_class=rec, macro_recall=macro, red_pct=100*(P==2).mean()), P
print(f"true red area (ground truth): {100*(Y[te]==2).mean():.1f}%\n")
print("SCENARIO 1: train on REAL daylight -> test on REAL daylight")
r1, P1 = run(Xg[tr], Y[tr], Xg[te], Y[te])
print(f" red recall {r1['per_class'][2]:.2f} | macro {r1['macro_recall']:.3f} | reports {r1['red_pct']:.1f}% red\n")
print("SCENARIO 2: train on REAL daylight -> test on REAL indoor")
r2, P2 = run(Xg[tr], Y[tr], Xb[te], Y[te])
print(f" red recall {r2['per_class'][2]:.2f} | macro {r2['macro_recall']:.3f} | reports {r2['red_pct']:.1f}% red\n")
print("SCENARIO 3: train on REAL daylight + REAL indoor -> test on REAL indoor")
r3, P3 = run(np.vstack([Xg[tr],Xb[tr]]), np.concatenate([Y[tr],Y[tr]]), Xb[te], Y[te])
print(f" red recall {r3['per_class'][2]:.2f} | macro {r3['macro_recall']:.3f} | reports {r3['red_pct']:.1f}% red")
print("\nfull per-class recall:")
for name, r in [('daylight->daylight', r1), ('daylight->indoor', r2), ('mixed->indoor', r3)]:
print(f" {name:20s} green {r['per_class'][0]:.2f} transition {r['per_class'][1]:.2f} red {r['per_class'][2]:.2f}")
true red area (ground truth): 75.4% SCENARIO 1: train on REAL daylight -> test on REAL daylight red recall 1.00 | macro 0.989 | reports 75.5% red SCENARIO 2: train on REAL daylight -> test on REAL indoor red recall 0.12 | macro 0.439 | reports 11.0% red SCENARIO 3: train on REAL daylight + REAL indoor -> test on REAL indoor red recall 0.96 | macro 0.669 | reports 80.9% red full per-class recall: daylight->daylight green 0.99 transition 0.98 red 1.00 daylight->indoor green 0.37 transition 0.83 red 0.12 mixed->indoor green 0.37 transition 0.68 red 0.96
7. Visualize the three prediction maps¶
Red = model says red. Yellow = transitioning. Green = still green.
Note: because alignment is not perfect (see the IoU score in step 3), the thin green sliver on the leaf is especially sensitive to small misalignment. Treat the red numbers as the reliable headline result.
def predmap(P):
out = np.full((*day.shape[:2], 3), 255, np.uint8)
col = {0:(0,180,0), 1:(0,200,255), 2:(0,0,220)}
m = np.full(day.shape[:2], -1, np.int64)
m[ys[te], xs[te]] = P
for k, v in col.items(): out[m==k] = v
out[~leaf] = (255,255,255)
return out
show(predmap(P1), f"daylight -> daylight (red recall {r1['per_class'][2]:.2f})")
show(predmap(P2), f"daylight -> real indoor (red recall {r2['per_class'][2]:.2f})")
show(predmap(P3), f"mixed -> real indoor (red recall {r3['per_class'][2]:.2f})")
8. Save and download your figures¶
cv2.imwrite("real_pred_good.png", predmap(P1))
cv2.imwrite("real_pred_goodbad.png", predmap(P2))
cv2.imwrite("real_pred_mixed.png", predmap(P3))
for f in ["real_pred_good.png", "real_pred_goodbad.png", "real_pred_mixed.png"]:
files.download(f)
What the Model Learned in Each Scenario
Daylight → The model learned a clean, simple rule: strong saturation plus a warm hue means red, weak saturation plus a warm hue means still turning, cool hue means green. Under daylight this rule is nearly perfect because the color separation between classes is wide and clean.
Dayligh, Real Indoor. The model applied that same daylight rule to a photo where the physical rule no longer held. Real indoor light didn't just dim the leaf, it collapsed the color separation the model depended on.
True-red pixels, now desaturated toward grey, landed in the same feature space the model had learned to call "green" or "transitioning." The model didn't fail randomly, it failed systematically and predictably, misreading nearly 9 out of 10 truly red pixels as something else. That is the worst possible kind of failure, because the model's confidence didn't drop, only its correctness did.
Mixed → Real Indoor. Once the model also saw examples of desaturated red during training, it learned a second rule specific to low light, and red recall snapped back to 0.96.
But transition recall fell from 0.98 to 0.68, and green recall never recovered at all, stuck at 0.37 in both indoor scenarios. That persistent green failure is not new information about lighting, it's the imperfect alignment (0.905 overlap, not 1.0) catching up with the smallest class.
Green is only 3.5% of the leaf, so a handful of misaligned pixels along that thin sliver do outsized damage to its score. This is a measurement artifact of the experiment, not a lighting effect, and it should be reported honestly as a limitation rather than folded into the headline finding.
Key Takeaways
The core hypothesis holds, and holds harder on real data than on simulated data. Red recall collapsed from 1.00 to 0.12 on your actual photos, worse than the 0.24 the simulation predicted. Real diffused indoor light is messier and more damaging than any synthetic approximation of it.
Mixing in bad data during training is a real, working fix, not a full one. Recovery brought red recall to 0.96 and the reported percentage back to 80.9%, close to the true 75.4%. But macro recall across all three classes only reached 0.67, well short of the 0.99 ceiling, because gains in one class came with losses in another.
A model's overall accuracy can hide a catastrophic single-class failure. In scenario two, red recall was 0.12 while transition recall stayed at a deceptively reassuring 0.83. Anyone watching only an averaged accuracy number would have missed that the model had essentially stopped detecting red at all.
Real-world experiments carry a cost simulations don't: measurement noise of their own. The alignment step introduced its own source of error, visible in the green class staying broken even after the main fix worked. A rigorous real-data experiment has to separate "the lighting effect we're studying" from "the noise our own method introduced," and here those two are cleanly separable: red is the trustworthy signal, green is the method's own limitation showing through.