The right words.

A RAG pipeline includes three stages: (1) ingestion, which converts documents into vector chunks; (2) retrieval, which fetches the chunks most relevant to the user's query; (3) generation, in which the LLM receives the chunks as context and produces an answer. RAG drastically reduces hallucinations and allows knowledge to be updated without retraining the model.

Modern LLMs (Qwen, Llama, Mistral, GPT, Claude) range from a few billion to hundreds of billions of parameters. Quantized 7-14B parameter models can run on consumer hardware (RTX 4090, Mac Studio) with performance adequate for enterprise use cases.

Common formats: GGUF (for llama.cpp, optimized for mixed CPU/GPU), AWQ (Activation-aware Weight Quantization, excellent for GPU), GPTQ. A 14B model in FP16 requires ~28 GB VRAM; quantized to 4-bit it drops to ~9 GB. The typical quality loss is below 2% on standard benchmarks.

Embeddings are the heart of semantic retrieval: models such as BGE-M3 (multilingual), E5, Cohere embed-v3 convert sentences and documents into vectors that can be indexed in a vector database and searched by cosine similarity. Specific multilingual embeddings (e.g. BGE-M3) perform much better on Italian corpora than English embeddings.

Mature solutions: Qdrant (open source, written in Rust, excellent for on-premise), pgvector (PostgreSQL extension, perfect if you already have Postgres), Weaviate, Milvus. For knowledge bases under 10M vectors pgvector is often sufficient; beyond that, Qdrant offers superior performance and features (pre-search filtering, hybrid search).

The typical pattern is retrieve-top-k=20 → re-rank → final top-5 to the context. It costs 100-300ms per query but improves retrieval precision by 15-25% in our internal benchmarks. Recommended models: BGE-reranker-large (open, Italian supported), Cohere Rerank.

Modern inference engines: vLLM (excellent throughput on GPU, continuous batching support), Ollama (excellent developer experience, lightweight), llama.cpp (mixed CPU/GPU, Apple Silicon support), TensorRT-LLM (maximum NVIDIA performance, high setup complexity).

MCP defines a client-server interface in which an LLM (client) can discover and invoke tools exposed by MCP servers (databases, APIs, file systems). It is becoming the de facto standard for reliable AI agents: it allows the model to be decoupled from the tools and centralized guardrails to be applied.

Local-first does not only mean 'offline': it means that the data perimeter coincides with the client's physical/legal perimeter. Advantages: data sovereignty, simplified GDPR compliance, zero marginal costs per query, predictable latency. Cost: initial hardware investment and greater management complexity.

For AI, edge computing translates into devices such as NVIDIA Jetson, a Mac Studio in a network cabinet, mini-PCs with NPU. It enables latencies below 100ms and operability even with degraded connectivity. Typically paired with a 4-bit quantized model.

For AI projects, the GDPR requires: a legal basis for processing, minimization, the right of access/erasure, a DPIA for high-risk processing. Local AI simplifies compliance because it avoids extra-EU transfers and reduces the attack surface. For special category data (art. 9, e.g. health data) a DPIA is almost always mandatory.

The AI Act distinguishes: unacceptable risk (prohibited), high risk (stringent obligations: technical documentation, conformity assessment, human oversight), limited risk (transparency), minimal. General-purpose generative models (GPAI) have specific documentation and copyright obligations. Phased entry into force 2025-2027.

AgID issues: AI Guidelines for the PA, the Interoperability Model, design guidelines for digital services, minimum ICT security measures. To provide services to the PA it is almost always necessary to be compliant with the CAD (Digital Administration Code) and registered on MePA.

For Italian organizations it is particularly relevant with respect to the U.S. CLOUD Act, which obliges U.S. providers to supply data to American authorities even if hosted in the EU. Data sovereignty is achieved with on-premise, edge, or certified sovereign cloud infrastructures (e.g. the national strategic hub).

Mandatory for the Italian PA (Stanca Act, law 4/2004) and for most European public websites (European Accessibility Act). It covers color contrast, keyboard navigation, screen readers, semantic structure. We are following the adaptation to WCAG 2.2 AA for projects from 2026 onwards.

Main languages: Solidity (Ethereum and EVM-compatible), Rust (Solana, NEAR), Move (Aptos, Sui). Institutional use cases: document notarization, transparent public registers, provenance certification. Limitation: once deployed they are difficult to modify — bug = permanent exploit.