AST Query System

ORMBridge's core power lies in its ability to translate client-side queries into backend-specific operations without tightly coupling your frontend to a particular ORM. This is accomplished through an Abstract Syntax Tree (AST) query system - a flexible intermediate representation of query operations that can be translated to different backend implementations.

Overview

When you write a query in the ORMBridge client like:

User.objects.filter({ username__contains: 'john' }).orderBy('-created_at')

Instead of directly generating SQL or ORM-specific code, ORMBridge:

1. Converts this into an AST representation
2. Sends this AST representation to the server
3. The server parses and executes this AST against the actual ORM (Django, SQLAlchemy, etc.)
4. Returns the results to the client

This abstraction layer allows your frontend code to remain unchanged even if you switch your backend ORM.

AST Structure

At its core, the AST consists of a tree of query nodes. Each node represents an operation with a specific type, plus any necessary parameters for that operation.

Node Types

The base structure of a node in the AST looks like:

interface QueryNode {
  type: QueryOperationType; // e.g., 'filter', 'or', 'and', 'not', etc.
  conditions?: Record<string, any>; // For direct field conditions
  children?: QueryNode[]; // For nested operations
  // Various operation-specific properties:
  lookup?: any;
  defaults?: Partial<any>;
  pk?: number;
  data?: any;
}

Common node types include:

Type	Description	Example
filter	Apply conditions to narrow results	{ type: 'filter', conditions: { name: 'John' } }
and	Combine multiple conditions with AND logic	{ type: 'and', children: [...nodes] }
or	Combine multiple conditions with OR logic	{ type: 'or', children: [...nodes] }
not	Negate the contained conditions	{ type: 'not', children: [...nodes] }
get	Retrieve a single object	{ type: 'get', conditions: { id: 1 } }
create	Create a new object	{ type: 'create', data: { name: 'John' } }
update	Update existing objects	{ type: 'update', filter: {...}, data: {...} }
delete	Delete objects	{ type: 'delete', filter: {...} }

Query Building Example

Let's see how a complex client query becomes an AST:

// Client query
const query = User.objects
  .filter({
    username__contains: 'john',
    Q: [
      Q('OR', { age__gte: 21 }, { verified: true })
    ]
  })
  .exclude({ role: 'admin' })
  .orderBy('-created_at');

// Resulting AST (simplified)
const ast = {
  filter: {
    type: 'and',
    children: [
      {
        type: 'filter',
        conditions: {
          username__contains: 'john'
        }
      },
      {
        type: 'or',
        children: [
          {
            type: 'filter',
            conditions: {
              age__gte: 21
            }
          },
          {
            type: 'filter',
            conditions: {
              verified: true
            }
          }
        ]
      },
      {
        type: 'not',
        children: [
          {
            type: 'filter',
            conditions: {
              role: 'admin'
            }
          }
        ]
      }
    ]
  },
  orderBy: [
    {
      field: 'created_at',
      direction: 'desc'
    }
  ]
};

The Server-Side Visitor Pattern

On the server, ORMBridge uses the Visitor pattern to traverse this AST and translate it into ORM-specific queries. Each backend adaptor implements a visitor that knows how to process each node type.

The Django Implementation

For example, the Django implementation uses the QueryASTVisitor class to convert AST nodes into Django's Q objects:

class QueryASTVisitor:
    def visit(self, node: Dict[str, Any]) -> Q:
        node_type: str = node.get("type")
        method = getattr(self, f"visit_{node_type}", None)
        if not method:
            raise ValidationError(f"Unsupported AST node type: {node_type}")
        return method(node)
        
    def visit_filter(self, node: Dict[str, Any]) -> Q:
        # Convert simple field conditions into Django Q objects
        q = Q()
        conditions: Dict[str, Any] = node.get("conditions", {})
        for field, value in conditions.items():
            # Parse field lookups like field__contains
            parts = field.split("__")
            if parts[-1] in self.SUPPORTED_OPERATORS:
                operator = parts.pop(-1)
            else:
                operator = "eq"
            lookup = "__".join(parts) if operator == "eq" else "__".join(parts + [operator])
            q &= Q(**{lookup: value})
        return q
        
    def visit_and(self, node: Dict[str, Any]) -> Q:
        return self._combine(node.get("children", []), lambda a, b: a & b)
        
    def visit_or(self, node: Dict[str, Any]) -> Q:
        return self._combine(node.get("children", []), lambda a, b: a | b)

Benefits of the AST Approach

1. Backend Agnostic: The client doesn't need to know anything about the backend ORM
2. Type Safety: The complete query structure is serialized, not just strings, preserving type information
3. Extensibility: New query capabilities can be added by extending the AST format
4. Security: Queries are structured and validated, reducing the risk of injection attacks
5. Performance Optimization: The backend can analyze the complete query AST and optimize execution

How It Works With Your Application

The flow of a query through the ORMBridge system:

1. Client builds a query using the ORMBridge TypeScript client
2. Client serializes the query as an AST and sends it to the server
3. Server receives the AST and identifies the target model
4. The appropriate ORM adapter's visitor processes the AST
5. The query is executed against the database
6. Results are serialized and returned to the client
7. Client deserializes the response into model instances

This architecture lets you write intuitive, type-safe queries on the frontend that automatically translate to efficient database operations on your backend - regardless of which ORM you're using.

Extending the AST System

To add custom operations to the AST system, you'll need to:

1. Define the new node type in the AST structure
2. Update each backend's visitor to handle the new node type
3. Add the corresponding client-side methods

This is typically only needed for very specialized query needs beyond what ORMBridge already provides.