Tangent/18/26: Rental Architecture & Systems Overview

0. Platform Interactivity Chart

Major Platform Interactivity Profiles by Day:

1. Core Architecture (The 4 Pillars)

• Tier 1: Hard Eliminations (Boolean Pass/Fail): Strict drop logic. Listings are instantly purged for geographic bounding box violations, explicitly stated pet bans, male-only restrictions, image fraud, or poor environmental lighting.
• Tier 2: Weighted Scoring: Surviving listings are scored based on preference variables (e.g., commute distance, window size). Note: The exact mathematical weights for these variables are the main unfinished component I need your help finalizing.
• The "Hold" Queue (Null/Missing Data Handling): Listings with missing critical information (e.g., unclear pet rules, missing images, vague locations) are not forced into pass/fail. They are isolated in a hold queue for manual review.
• The Hybrid Summary (Clean Output): The final deliverable must be a highly readable summary showing Tier 1 status, Tier 2 score, Hold status, and a brief explanation. No raw scraping dumps or AI-generated prose.

2. Critical Operational Rules

• Visual Evidence Override: If text descriptions and images conflict, images win. The system requires basic image intelligence integration to verify features (e.g., AC types, window presence) and identify duplicate photos.
• Price is a Hard Ceiling, Not a Sort Variable: Price functions purely as a < $X gateway. A $100 price difference below the ceiling does not alter a listing's score.
• Historical State Tracking: The system must utilize a database to track listing persistence over time, identifying price drops, duplicate reposts, and broader market patterns rather than just scraping single-day snapshots.

3. Target Platforms

Apartments.com, AppFolio, Avail, Buildium, Chicago Domu, Cozy, Craigslist, Facebook Marketplace, Homes.com, HotPads, LA RentHop, Local MLS feeds, Miami Apartments Guide, Nextdoor Rentals, NYC StreetEasy, PadMapper, Reddit Housing Subs, Redfin, Redfin Rentals, Realtor.com, Realtor.com Rentals, Rent.com, RentCafe, Trulia, Trulia Rentals, TurboTenant, Zillow, Zillow Rentals, Zillow Rental Manager.

4. Suggested Scraper Data Schema

source_site
source_id
scrape_timestamp
url
title
price
currency
bedrooms
bathrooms
square_feet
address (raw)
lat_lon (if available)
image_urls
image_hashes
pet_policy
gender_restrictions
description_text
posting_date
contact_info
tier1_pass (boolean)
tier1_reasons (text)
tier2_score (numeric)
hold_flag (boolean)
manual_review_notes
history_id (link to time-series record)
      

5. Site Scraping & Defense Matrix

6. Four Types of Image Intelligence

1. Binary Feature Detection: Yes/No presence test (e.g., Are windows present? Is AC visible?). Ideal for Tier 1 eliminations.

2. Object Classification: Categorizes specific types (e.g., Window AC vs. Minisplit vs. Portable AC).

3. Attribute Extraction: Measures attributes and sizes (e.g., Large vs. Small window, Bright vs. Dim room lighting).

4. Scene Understanding: Identifies total context and room type (e.g., verifying if a windowless room is actually a bedroom).

7. Scarcity-Adaptive Weighting System

Increases core survival weights when weekly listings ($L$) drop low.

Scarcity Factor formula: $S = 1.5$ if $L \le 10$; $S = 1.2$ if $10 < L \le 20$; $S = 1.0$ if $L > 20$.

Final Score: Tier2Score = S * (Window + AC + Commute + Safety) + 0.5 * SecondaryScore

8. Stability-First Weighting System

Prioritizes stability variables: Price stability ($P_s$), Listing persistence ($L_a$), Price Volatility ($V$), and Management Reputation ($M$).

Formula: RawScore = 0.28(Ps) + 0.22(La) + 0.20(V) + 0.20(M) - 0.10(RepostPenalty) + 0.20(Quality & Disclosure)

9. Light Quality & Ventilation Weighting System

Weights natural light, window count/size, ventilation, and AC type.

Formula: R = 0.30(WindowScore) + 0.30(NaturalLight) + 0.20(Ventilation) + 0.20(AC_Score - Penalty)

Normalized Score: Tier2Score = 100 * (R / 3)