FactorsAiAutomation/ReadMe.md

# Factors.AI — Account Export Automation
### Automatic daily email alerts when new companies visit your website

---

## What does this tool do?

Every time you run it, this tool does **four things automatically** — no clicking, no manual work:

```
1. LOGS IN   → Opens Factors.AI in the background and signs in with your credentials
2. DOWNLOADS → Goes to Account Profiles, clicks the export button, downloads the CSV
3. COMPARES  → Checks: are there any new company domains that weren't in the last export?
4. EMAILS    → If yes, sends a formatted HTML email to everyone on your list
               If no new entries → does nothing (no email sent)
```

After every run, the fresh CSV is saved as `data/last_export.csv` — that becomes the baseline for the next comparison.

---

## What the email looks like

**Subject line examples:**

| Situation | Subject |
|---|---|
| Very first run ever | `[Factors.AI] First Run — 483 Accounts Exported (2026-07-01)` |
| New companies found | `[Factors.AI] 7 New Accounts Detected (2026-07-01)` |
| Nothing new | *(no email sent at all)* |

**Email body:** A styled HTML table with one row per new company, showing:

| Account Domain | Company Name | Company Industry | Company Employee Range | Company Annual Revenue | Last Activity |
|---|---|---|---|---|---|
| dell.com | Dell | Internet Software & Services | 100K+ | $102,300,000,000 | 2026-07-01 04:48:32 |
| adobe.com | Adobe | Internet Software & Services | 10K-50K | $19,409,000,000 | 2026-06-18 11:29:45 |

---

## How "new entry" is decided

The tool uses **Account Domain** (the website address, e.g. `dell.com`) as the unique identifier for each company.

- A company is **new** if its domain appears in today's export but was **not** in `data/last_export.csv`
- The comparison is case-insensitive (`Dell.com` = `dell.com`)
- Only `data/last_export.csv` is kept — no growing archive, no extra storage

---

## Files in this folder

```
factors_automation/
│
├── main.py           ← The only file you ever run ("python main.py")
├── scraper.py        ← Controls the Chrome browser (login, navigate, download)
├── comparator.py     ← Compares today's CSV with last_export.csv
├── mailer.py         ← Builds the HTML email and sends it via SMTP
├── config.py         ← All your settings live here (reads from .env)
│
├── requirements.txt  ← List of Python packages the tool needs
├── .env.example      ← Template — copy this to .env and fill in your details
├── .env              ← YOUR private settings (create this yourself — never share it)
│
├── data/
│   └── last_export.csv   ← Auto-created after first run. This is the comparison baseline.
│
├── tmp_download/         ← Temporary staging folder. Chrome downloads here first.
│                            Cleaned automatically before each run.
│                            Error screenshots are saved here if something goes wrong.
│
└── automation.log        ← Full log of every run (auto-created). Check this if anything fails.
```

> **Do not delete** `data/last_export.csv` — it is the memory of the tool.
> If you delete it, the next run will treat every account as new and email the full list.

---

## Prerequisites — what you need before setup

You need **three things** installed on the computer that will run this:

### 1. Python 3.10 or newer
Check your version by opening a terminal and typing:
```
python --version
```
If it shows `Python 3.10.x` or higher, you're good.
If not, download from: https://www.python.org/downloads/

### 2. Google Chrome browser
The tool controls Chrome in the background. Download from: https://www.google.com/chrome/

> The matching ChromeDriver (what lets Python control Chrome) is downloaded **automatically** the first time you run the tool. You do not need to install it yourself.

### 3. A Gmail account to send from (recommended)
Any Gmail account works. You will need to generate an **App Password** for it (explained in the setup below). You cannot use your regular Gmail password.

---

## Setup — step by step

### Step 1 — Download and unzip the project

Unzip `factors_automation.zip` somewhere on your computer, for example:
- **Mac/Linux:** `~/factors_automation/`
- **Windows:** `C:\factors_automation\`

### Step 2 — Open a terminal in that folder

- **Mac:** Right-click the folder → "New Terminal at Folder"
- **Windows:** Open the folder in File Explorer → click the address bar → type `cmd` → press Enter

### Step 3 — Create a virtual environment

A virtual environment keeps this tool's packages separate from everything else on your computer.

```bash
python -m venv venv
```

Then activate it:

```bash
# Mac / Linux:
source venv/bin/activate

# Windows:
venv\Scripts\activate
```

You will see `(venv)` appear at the start of your terminal prompt. This means it is active.

### Step 4 — Install the required packages

```bash
pip install -r requirements.txt
```

This downloads and installs four packages:
- **selenium** — controls the Chrome browser
- **webdriver-manager** — automatically downloads the right ChromeDriver
- **pandas** — reads and compares CSV files
- **python-dotenv** — reads your settings from the `.env` file

### Step 5 — Create your `.env` settings file

Copy the template:

```bash
# Mac / Linux:
cp .env.example .env

# Windows:
copy .env.example .env
```

Now open the `.env` file in any text editor (Notepad, TextEdit, VS Code, etc.) and fill in your real values. Here is what each line means:

```
FACTORS_EMAIL=natesh.krishnan@yourcompany.io
```
The email address used to log in to app.factors.ai.

```
FACTORS_PASSWORD=YourActualPassword
```
The password for that Factors.AI account.

```
SMTP_HOST=smtp.gmail.com
```
Leave this as-is if you are using Gmail. Change only if you use a different email provider (Outlook = `smtp.office365.com`, etc.).

```
SMTP_PORT=587
```
Leave this as-is. Port 587 is the standard secure email port (STARTTLS). Works with all major email providers.

```
SENDER_EMAIL=your_sender@gmail.com
```
The Gmail address the alerts will be sent **from**.

```
SENDER_PASSWORD=xxxx xxxx xxxx xxxx
```
**This is NOT your regular Gmail password.** You must generate an **App Password** — see the section below.

```
RECIPIENT_EMAILS=person1@company.com,person2@company.com
```
Everyone who should receive the alert emails. Separate multiple addresses with commas. No spaces around the commas.

```
HEADLESS=true
```
`true` means Chrome runs invisibly in the background (recommended for scheduled/daily runs).
`false` means you can watch Chrome open and do its work on screen (useful when testing or debugging).

---

### How to generate a Gmail App Password

Gmail does not allow scripts to log in with your regular password. You must create a special one-time App Password.

1. Go to your Google Account: https://myaccount.google.com/
2. Click **Security** in the left sidebar
3. Under "How you sign in to Google", click **2-Step Verification** and enable it if not already on
4. Go back to Security → scroll down → click **App Passwords**
5. Under "Select app" choose **Mail** → under "Select device" choose **Other** → type `FactorsAI Bot`
6. Click **Generate**
7. Google shows a 16-character code like `abcd efgh ijkl mnop`
8. Copy it (including spaces) into your `.env` file as `SENDER_PASSWORD`

> This App Password only lets the script send emails. It cannot access your Gmail inbox or change your account.

---

## Choosing which columns to export

By default (`FIELDS_TO_SELECT = None` in `config.py`), the tool keeps whichever columns are **already checked** in the Factors.AI export modal — which is the default 5:

- Company Name
- Company Industry
- Company Employee Range
- Company Annual Revenue
- Last Activity

The downloaded CSV will also always include **Account Domain** as the first column (this is always exported by Factors.AI and is used as the unique comparison key).

### To change which columns are exported

Open `config.py` in a text editor. Find this section near the bottom:

```python
FIELDS_TO_SELECT: list[str] | None = None   # None = use defaults
```

Change it to a list of the fields you want. For example, to export only Company Name, Industry, and Last Activity:

```python
FIELDS_TO_SELECT = [
    "$6Signal_name",
    "$6Signal_industry",
    "last_activity",
]
```

**Full list of available field codes:**

| Code | Column name in CSV |
|---|---|
| `"$6Signal_name"` | Company Name |
| `"$6Signal_industry"` | Company Industry |
| `"$6Signal_employee_range"` | Company Employee Range |
| `"$6Signal_annual_revenue"` | Company Annual Revenue |
| `"last_activity"` | Last Activity |
| `"$tag_hidden"` | Tags Hidden |
| `"$latest_source"` | Account Latest Source |
| `"$latest_campaign"` | Account Latest Campaign |
| `"$initial_campaign"` | Account First Campaign |
| `"$account_activity_url"` | Account Activity URL |
| `"$domain_name"` | Company ID |
| `"$latest_page_url"` | Account Latest Page URL |

When you set `FIELDS_TO_SELECT` to a list, the tool first clicks "Clear All" in the modal, then checks exactly the fields you listed. Setting it back to `None` restores the default behaviour.

---

## Running the tool

Make sure `(venv)` is active in your terminal before running.

### Normal run (invisible browser, sends real email)
```bash
python main.py
```

### Watch mode — opens real Chrome window (good for testing)
```bash
python main.py --visible
```
Use this the first time you run it to verify everything works correctly.

### Dry run — downloads and compares, but does NOT send any email
```bash
python main.py --dry-run
```
The new entries are printed to the terminal instead. Useful for checking what would have been emailed.

### Combine both flags
```bash
python main.py --visible --dry-run
```

---

## What happens during a run (step by step)

When you run `python main.py`, here is exactly what happens internally:

**Step 1 — Download CSV**
- Chrome launches (invisibly unless `--visible`)
- Opens `https://app.factors.ai/`
- Enters email and password from your `.env`
- Waits up to 30 seconds for login to complete
- Finds and clicks the Account Profiles page in the sidebar navigation. If that fails, it tries five known URL patterns (`/accounts`, `/account-profiles`, `/accounts/profiles`, `/analytics/accounts`, `/v2/accounts`)
- Clicks the download (↓) toolbar icon
- The export modal appears — columns are selected (or kept as default)
- Clicks "Export CSV"
- Waits up to 90 seconds for the file to appear in `tmp_download/`
- Chrome closes

**Step 2 — Compare**
- Loads `tmp_download/<downloaded>.csv` and `data/last_export.csv`
- Compares every row's **Account Domain** (first column)
- Identifies domains in the new file that were not in the old file
- If `data/last_export.csv` does not exist → all rows are treated as new (first run)

**Step 3 — Email** *(skipped if zero new entries and not first run)*
- Builds an HTML email with a table of new rows
- Connects to `smtp.gmail.com:587` via STARTTLS
- Logs in with `SENDER_EMAIL` and `SENDER_PASSWORD`
- Sends the email to all `RECIPIENT_EMAILS`

**Step 4 — Save**
- Copies the downloaded CSV to `data/last_export.csv`, replacing the previous one
- This becomes the new baseline for the next run

Everything is logged to `automation.log` in real time.

---

## Scheduling — run automatically every day

### Mac / Linux (using cron)

Open the cron editor:
```bash
crontab -e
```

This runs every day at 01:30 UTC, which is 07:00 IST. (adjust the paths to match your actual folder):
```
30 1 * * * /full/path/to/factors_automation/venv/bin/python /full/path/to/factors_automation/main.py
```

To find your full path, run this command inside the project folder:
```bash
pwd
```

Example result: `/home/natesh/factors_automation`

So the cron line would be:
```
0 8 * * * /home/natesh/factors_automation/venv/bin/python /home/natesh/factors_automation/main.py
```

### Windows (using Task Scheduler)

1. Open **Task Scheduler** (search for it in the Start menu)
2. Click **Create Basic Task** on the right
3. Name: `Factors.AI Export`
4. Trigger: **Daily** at your preferred time
5. Action: **Start a program**
6. Program: `C:\factors_automation\venv\Scripts\python.exe`
7. Arguments: `C:\factors_automation\main.py`
8. Click **Finish**

---

## Checking if a run succeeded

Open `automation.log` in any text editor. A successful run looks like this:

```
2026-07-01 08:00:01  [INFO    ]  __main__ — =================================================================
2026-07-01 08:00:01  [INFO    ]  __main__ —   Factors.AI Account Export Automation  —  START
2026-07-01 08:00:01  [INFO    ]  __main__ —   headless=True   dry_run=False
2026-07-01 08:00:01  [INFO    ]  __main__ — =================================================================
2026-07-01 08:00:01  [INFO    ]  __main__ — STEP 1: Downloading CSV …
2026-07-01 08:00:04  [INFO    ]  scraper — Opening https://app.factors.ai/ …
2026-07-01 08:00:09  [INFO    ]  scraper — Email entered.
2026-07-01 08:00:10  [INFO    ]  scraper — Password entered.
2026-07-01 08:00:11  [INFO    ]  scraper — Login form submitted.
2026-07-01 08:00:14  [INFO    ]  scraper — Logged in successfully. URL: https://app.factors.ai/accounts
2026-07-01 08:00:18  [INFO    ]  scraper — Download button clicked.
2026-07-01 08:00:21  [INFO    ]  scraper — 'Export CSV' button clicked.
2026-07-01 08:00:24  [INFO    ]  scraper — Download complete → tmp_download/accounts.csv
2026-07-01 08:00:24  [INFO    ]  scraper — Browser closed.
2026-07-01 08:00:24  [INFO    ]  __main__ — STEP 2: Comparing with previous export …
2026-07-01 08:00:24  [INFO    ]  comparator — Comparison complete: 7 new / 490 total (previous had 483 rows)
2026-07-01 08:00:24  [INFO    ]  __main__ —   ✓ New entries detected (7 new)
2026-07-01 08:00:24  [INFO    ]  __main__ — STEP 3: Sending alert email …
2026-07-01 08:00:26  [INFO    ]  mailer — Email sent to: ['you@company.com']  |  subject: [Factors.AI] 7 New Accounts Detected (2026-07-01)
2026-07-01 08:00:26  [INFO    ]  __main__ — STEP 4: Saving export as last_export.csv
2026-07-01 08:00:26  [INFO    ]  __main__ — =================================================================
2026-07-01 08:00:26  [INFO    ]  __main__ —   Automation complete ✓
```

---

## Troubleshooting

### "Could not find email input on the login page"
The tool took a screenshot. Open `tmp_download/login_page_not_found.png` to see what the browser was looking at.

Possible causes:
- Factors.AI is down or showing a maintenance page
- Your internet connection is slow (increase `_PAGE_LOAD_TIMEOUT` in `scraper.py` from `30` to `60`)
- The login page layout changed (contact whoever maintains this tool)

### "Still on login page after submission — check credentials"
Your `FACTORS_EMAIL` or `FACTORS_PASSWORD` in `.env` is wrong. Check and correct them.
Screenshot saved as `tmp_download/post_login_timeout.png`.

### "Could not navigate to the Account Profiles page"
The tool could not find the Accounts section. Screenshot saved as `tmp_download/accounts_not_found.png`.

Try running with `--visible` to watch what happens:
```bash
python main.py --visible --dry-run
```

### "CSV download did not complete within 90 seconds"
The download took too long. Possible causes:
- Very large export (many thousands of rows)
- Slow internet

Open `scraper.py`, find the line `_DOWNLOAD_TIMEOUT = 90` near the top, and change `90` to `180`.

### Gmail error: "Username and Password not accepted"
You used your regular Gmail password instead of an App Password.
Follow the **"How to generate a Gmail App Password"** steps above.

### Gmail error: "SMTPAuthenticationError"
Same as above, or 2-Step Verification is not enabled on the Gmail account.

### `SyntaxError` or `python: command not found`
Your Python version is older than 3.10. Check with `python --version` and upgrade at https://www.python.org/downloads/

### The `(venv)` prefix disappeared from my terminal
Your virtual environment is no longer active. Reactivate it:
```bash
# Mac / Linux:
source venv/bin/activate

# Windows:
venv\Scripts\activate
```

---

## Security notes

- Your `.env` file contains passwords. **Never share it, email it, or commit it to Git.**
- The `.env.example` file contains only placeholder text and is safe to share.
- The Gmail App Password only allows the script to send email. It cannot read your inbox, delete messages, or change your Google account settings.
- `data/last_export.csv` contains company names and domains from Factors.AI. Treat it with the same care as any customer data.

---

## Quick-reference card

| Task | Command |
|---|---|
| Normal run (background, sends email) | `python main.py` |
| Watch browser work on screen | `python main.py --visible` |
| Test without sending email | `python main.py --dry-run` |
| Watch + test, no email | `python main.py --visible --dry-run` |

| File | Purpose |
|---|---|
| `.env` | Your private credentials and settings |
| `config.py` | Export field selection and advanced settings |
| `data/last_export.csv` | Comparison baseline — do not delete |
| `automation.log` | Full run history — check when something fails |
| `tmp_download/*.png` | Error screenshots — check when a step fails |