AI Automation Framework

A lightweight, program-controlled(Harness) AI automation framework built with Node.js. This framework allows AI to fill specific parts of a workflow while maintaining strict program validation and control.

Features

• Program-Controlled Workflow: AI helps plan and fill parameters, but execution is controlled by program code
• Restricted Planning: Planning uses only known skill names from the skill registry
• Workflow Template Reuse: Approved/reusable workflows are cached in SQLite and reused on future matching requests
• Request Normalization: Variable parts in user requests such as URLs, emails, quoted strings, and numbers can be normalized into reusable templates
• Dynamic Skill Registry: Easily add new capabilities by dropping skill files into the src/skills/ directory
• State Machine: Robust task and step status management with SQLite persistence
• Inter-Step Data Flow: Previous step outputs automatically feed into later steps
• Local LLM Integration: Uses Ollama for AI capabilities
• Event Tracking: Comprehensive logging of task and step events
• Admin Workflow Control: Activate, disable, reject, list, or delete saved workflows from CLI

---

Core Idea

This project is designed around one main principle:

AI can help generate plans, but the program must remain the final controller of execution.

Instead of letting the LLM freely define the entire workflow at runtime every time, this framework:

1. Uses AI to select skills from a restricted skill set
2. Validates the returned plan in code
3. Executes each step in code
4. Saves reusable workflow templates into SQLite
5. Reuses known-good workflow templates next time without calling the LLM again for planning

This reduces randomness and improves repeatability.

---

Architecture

Core Components

• Task Engine: Orchestrates planning → step building → execution
• Skill Registry: Dynamically loads atomic skills from src/skills/
• LLM Provider: Handles communication with Ollama API
• Task Repository: SQLite persistence, task/step/event tracking, workflow template storage
• Prompt Builder: Builds prompts for planning and parameter filling
• Validators: Program-side validation of AI outputs before state transitions

Skill System

Skills are atomic, reusable capabilities. Each skill defines:

• stepName: Unique identifier
• requiresAI: Whether the skill needs LLM assistance
• description: Human-readable description
• payloadDefinition: Expected input payload fields
• execute(): Implementation function
• validate(): Output validation function

---

Workflow

1. Planning

The framework first tries to find an existing workflow template by normalized request template.

If found:

• Reuse the saved workflow directly if the workflow is active
• If the workflow is inactive, require human to approve or reject the template before proceeding

If not found:

• Call restrictedPlanningSolution
• Ask the LLM to select and order skills using only available skill names
• Save the newly created workflow template with status inactive by default when workflow.autoActivate is false

2. Building Steps

The framework asks the LLM to fill in the payload for each planned skill using:

• user raw input
• skill payload definition
• prior execution context

For a newly created workflow template, the framework also forms a structured workflow definition to record which parameters for each planned skill can be retrieved from the target input.

After human approval, this structured workflow becomes a guardrail for future requests that fit the same normalized template:

• it confirms the planned skill sequence is correct for this template
• it confirms which skill parameters should be retrievable from this kind of input
• if a future parameter extraction misses a parameter that the approved template says should be present, the framework can retry extraction up to workflow.fillParameter.maxFillParameterRetryCount times based on that guardrail

3. Execution

The program executes steps in order and validates results step by step.

4. Persistence

If a workflow was newly generated by restricted planning and passes validation, it is saved as a reusable workflow template. The default value of workflow.autoActivate is false, so the framework creates the workflow template on the first matching request and saves it with status inactive. When a later request fits the same template and the template is still inactive, the framework requires human to approve or reject the template before proceeding. Once approved, the template can be reused automatically for future matching requests. You can change the default behavior via config.json's workflow.autoActivate field.

5. Output

The final output includes:

• taskId
• workflowId
• workflowSource
• normalizedRequestTemplate
• plan
• steps

---

Workflow Template Reuse

A major part of this framework is workflow template caching.

Example:

User request:

Summarize page www.example.com then send to example@example.com

Normalized template:

Summarize page <URL> then send to <EMAIL>

If this normalized template has already been saved in the database with an active workflow, the framework can skip planning and directly reuse the workflow.

If an active workflow failed during the execution, it will be set to "inactive" status automatically. This can be configured in config.json's workflow.autoInactivate field.

Supported normalization examples

• URL → <URL>
• email → <EMAIL>
• quoted string → "<STRING>" or '<STRING>'
• number → <NUMBER>

Example:

Convert{id: 123, description: "item description"}into XML format

Normalized:

Convert{id: <NUMBER>, description: "<STRING>"}into XML format

This improves workflow reuse without relying on embeddings or fuzzy semantic matching.

---

Workflow Status

Saved workflows use the following statuses:

• active: can be reused automatically
• inactive: temporarily disabled from reuse
• rejected: marked as incorrect and should not be reused

You can also physically delete a workflow record if needed.

---

Installation

Prerequisites

• Node.js 18+
• Ollama
• SQLite3

Setup

1. Clone the repository:

git clone https://github.com/aotol/AI_AUTOMATION.git
cd AI_AUTOMATION

2. Install dependencies:

npm install

3. Pull your Ollama model:

ollama pull gemma4:e4b

4. Edit your configuration:

# update config.json as needed

---

Configuration

Edit config.json:

{
  "llm": {
    "provider": "ollama",
    "baseUrl": "http://localhost:11434",
    "model": "gemma4:e4b",
    "timeoutMs": 120000,
    "maxOutputTokens": 4000,
    "maxInputChars": 10000,
    "isReasoningModel": false,
    "reasoningStripMode": "auto"
  },
  "sqlite": {
    "path": "data/tasks.db"
  },
  "input": {
    "maxLength": 10000
  },
  "output": {
    "maxLength": 50000
  },
  "workflow": {
    "autoActivate": false,
    "autoInactivate": true,
    "fillParameter": {
      "maxFillParameterRetryCount": 3
    }
  },
  "email": {
    "provider": "smtp",
    "smtp": {
      "name": "AI Automation Framework",
      "host": "smtp-host",
      "port": 465,
      "secure": true,
      "auth": {
        "user": "your-email@yahoo.com",
        "pass": "your-app-password"
      }
    },
    "imap": {
      "user": "your-email@yahoo.com",
      "pass": "your-app-password",
      "host": "imap-host",
      "port": 993,
      "tls": true
    }
  }
}

Email Setup Notes

• Use app-specific passwords for Email account
• For Gmail, enable 2FA and generate an app password
• The pass field is used for both SMTP and IMAP authentication

---

Basic Usage

Run a task from CLI:

node ./src/app.js "Fetch https://example.com, extract text, and summarize it"

Another example:

node ./src/app.js "Summarize www.example.com then email to example@example.com in Chinese"

The output will include workflow information, for example:

{
  "taskId": "task_123",
  "workflowId": 12,
  "workflowSource": "restricted_planning",
  "normalizedRequestTemplate": "Summarize <URL> then email to <EMAIL> in Chinese",
  "plan": {
    "fetch_url": ["url"],
    "extract_text_from_html": [],
    "summarize_text": [],
    "translate_text": ["targetLanguage"],
    "send_email": ["address", "subject"]
  },
  "steps": [
    {
      "stepName": "fetch_url",
      "output": {},
      "validation": {}
    }
  ]
}

If the same normalized request comes again later and the workflow is still active, the system can reuse it directly from SQLite. If the matching workflow is still inactive, the framework pauses and requires human approval or rejection before proceeding.

---

Admin Commands

List all workflows

node ./src/app.js admin list-workflows

Activate a workflow

node ./src/app.js admin activate-workflow 12

Disable a workflow

Internally this sets the workflow status to inactive.

node ./src/app.js admin disable-workflow 12

Reject a workflow

node ./src/app.js admin reject-workflow 12

Delete a workflow

node ./src/app.js admin delete-workflow 12

---

Programmatic Usage

const TaskEngine = require('./src/task-engine');
const TaskRepository = require('./src/task-repository');
const llmProvider = require('./src/llm-provider');
const promptBuilder = require('./src/prompt-builder');
const logger = require('./src/logger');
const { config } = require('./src/config');

(async function () {
  const taskRepository = new TaskRepository();
  await taskRepository.init();

  const services = {
    config,
    taskRepository,
    llmProvider,
    promptBuilder,
    logger
  };

  const engine = new TaskEngine(services);
  const result = await engine.runTask('Fetch https://example.com and summarize it');

  console.log(result);
})();

---

Example Workflow

For input:

Fetch https://example.com, extract text, and summarize it

Planning Phase

The framework may generate:

{"fetch_url":[],"extract_text_from_html":[],"summarize_text":[]}

Building Steps Phase

The framework may fill parameters like:

{
  "steps": [
    {
      "stepIndex": 0,
      "stepName": "fetch_url",
      "requiresAI": false,
      "payload": {
        "url": "https://example.com"
      }
    },
    {
      "stepIndex": 1,
      "stepName": "extract_text_from_html",
      "requiresAI": false,
      "payload": {}
    },
    {
      "stepIndex": 2,
      "stepName": "summarize_text",
      "requiresAI": true,
      "payload": {
        "maxLength": "200 words"
      }
    }
  ]
}

Execution Phase

The program runs each step in sequence and validates outputs.

Persistence Phase

If this plan is newly generated, the normalized request template, skill sequence, and structured parameter guardrail can be saved into approved_workflow_templates.

---

Adding New Skills

Create a new file in src/skills/, for example my_skill.js:

module.exports = {
  stepName: 'my_skill',
  requiresAI: false,
  payloadDefinition: {
    text: 'The input text for processing.',
    url: 'The URL to fetch.'
  },
  description: 'Description of what this skill does',
  execute: async function (context, services, stepDefinition) {
    return { result: 'output' };
  },
  validate: async function (context, result, stepDefinition) {
    const errors = [];

    return {
      valid: errors.length === 0,
      errors
    };
  }
};

The skill will be loaded automatically on next startup.

---

Skill payloadDefinition and dynamic payload assignment

• Each skill may define a payloadDefinition object
• buildFillSkillParameterPrompt() uses this to tell the LLM what parameters it should extract
• During step building, the LLM fills payload fields based on user input
• If a parameter is not available directly from user input, the step implementation can still derive it from prior step outputs

This keeps the system abstract and avoids hardcoding payload values for every case.

---

Built-in Skills

Examples of built-in skills include:

• fetch_url: downloads webpage content
• extract_text_from_html: extracts readable text from HTML
• detect_language: detects text language
• translate_text: translates text
• summarize_text: summarizes text
• receive_email: retrieves latest email from IMAP inbox
• send_email: sends email via SMTP
• search_web: performs web search
• format_output: formats final result

---

SQLite Tables

The framework uses SQLite for persistence.

tasks

Stores the top-level task record.

task_steps

Stores each step in execution order.

task_events

Stores detailed task events for debugging and auditing.

workflow_templates

Stores reusable workflow templates:

• raw_request
• normalized_request_template
• planned_skills
• source
• status
• created_at
• updated_at

---

Development

Project Structure

src/
├── app.js
├── config.js
├── input-provider.js
├── llm-provider.js
├── logger.js
├── prompt-builder.js
├── skill-utils.js
├── skills.js
├── task-engine.js
├── task-repository.js
├── validators.js
└── skills/
    ├── fetch_url.js
    ├── extract_text_from_html.js
    └── ...

Testing

Test module loading:

node -e "require('./src/skills'); console.log('Modules loaded successfully');"

Run a task:

node ./src/app.js "Test task description"

List workflows:

node ./src/app.js admin list-workflows

---

Troubleshooting

Ollama connection failed

Make sure Ollama is running and the configured URL is correct.

Skill not found

Make sure the skill file exists in src/skills/ and exports a valid stepName.

Database errors

Make sure the SQLite file path is writable.

---

Contributing

1. Fork the repository
2. Create a feature branch
3. Add or improve skills in src/skills/
4. Test thoroughly
5. Submit a pull request

---

License

MIT License - see LICENSE for details.