Skip to content

feat(effect-ai): GoogleLanguageModel over /google/v1 generateContent (phase 5) - #97

Merged
anandpant merged 4 commits into
mainfrom
effect/phase-5
Nov 20, 2026
Merged

anandpant merged 4 commits into
mainfrom
effect/phase-5

Conversation

@anandpant

@anandpant anandpant commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Phase 5 of packages/foundry-effect-ai/docs/PLAN.md: GoogleLanguageModel over Foundry's /google/v1/models/{rid}:generateContent and :streamGenerateContent?alt=sse, with internal/google-gemini/ (schema, request, response, stream) and internal/models/google.ts.

  • Bearer auth comes from FoundryClient; the model RID goes in the path. Aliases resolve through the Gemini model list, and any other target passes through.
  • config takes thinkingConfig, the generationConfig field of the same name, sent as given. Foundry accepts upper-case thinkingLevel values only; low returns 422.
  • Thought signatures stay under foundry-google on the part that carried them, including signature-only fragments with or without a text field, and go back on that part when replayed. In streams, a signature closes its text or reasoning block. Gemini rejects a replayed function call without its signature (400), so the tools cells exercise replay for real.
  • Function calls keep Gemini's wire id (Foundry always sent one). A call without one gets a local ID from IdGenerator, marked localId, which replay leaves off the functionCall and its functionResponse. Arguments arrive as objects and pass through to Toolkit unchanged.
  • Structured output, recorded in PLAN.md: responseJsonSchema from LanguageModel.defaultCodecTransformer. A live check accepted the default schema (additionalProperties, anyOf, integer bounds, enum, $defs/$ref) and the output conformed. const is accepted but ignored, and Effect never emits it, so no Gemini subset rewrite is needed. Keywords outside Gemini's subset and types with no JSON form pass through unchecked; the codec still validates the answer. PLAN.md says so.
  • JSON output with tools: with responseJsonSchema and tools in one request, gemini-3.1-flash-lite called the tool again after every result (16 of 16 tries). In that case the schema is sent as a synthetic json function under ANY mode, as @ai-sdk/google does for forced tool choice. Its call comes back as the only answer text (other text in that mode is dropped) and replays as the original signed functionCall. The json call answered 3 of 3 tries, and a live third turn replaying it succeeded.
  • Finish reasons: STOP → stop or tool-calls, MAX_TOKENS and CONTINUATION → length, safety reasons and a confirmed prompt block → content-filter, MALFORMED_FUNCTION_CALL → error. Gemini streams have no terminal event, so the finish is emitted at end of stream, and only after a finish reason (not empty or FINISH_REASON_UNSPECIFIED) or prompt block arrived; otherwise the stream fails with InvalidResponseError.
  • Streamed tool calls: parameter previews stream as they arrive, but the tool-call parts, and content after them, are held and emitted in one array with the finish. Core then skips handlers on a length or content-filter turn, and a truncated stream never starts one.
  • An in-band {error} (JSON body or SSE event) is classified by numeric code or RPC status: 400/404/409 → InvalidRequestError, 401/403 → AuthenticationError, 429 → RateLimitError, 5xx or none → InternalProviderError.
  • Usage: input is promptTokenCount + toolUsePromptTokenCount, output is candidates plus thoughts. A missing snapshot (Gemini sends {} until the last chunk) leaves usage undefined; inside a snapshot, missing counts are zero because proto3 JSON omits zeros.
  • Attribution: Gemini has no Effect provider, so the internal files say // Adapted from @ai-sdk/google (Apache-2.0), 4.0.122. and NOTICE credits Vercel. PLAN.md's upstream-code row says so.

Matrix

The GoogleLanguageModel row has no todos: text, stream, json, tools, stream-tools, json-tools, image, reasoning, stream-truncated, stream-error.

  • Cells run on gemini-3.1-flash-lite. Reasoning cells send thinkingConfig: { includeThoughts: true, thinkingLevel: 'HIGH' }: without a level flash-lite returns no thoughts, and at LOW streamed summaries were unreliable.
  • The recorded tools and stream-tools first turns carry a thought and a signed function call, and the second request replays both. The target's replay check asserts the signature on the call.
  • stream-truncated keeps only the first chunk of the recorded stream, before the finish reason. stream-error follows it with a Gemini {"error":{"code":500,"status":"INTERNAL"}} event.
  • Tree-shaking budget: 17,530 bytes measured, 21,000 set.

Checks

  • pnpm run verify passes.
  • Live, Google row only (-t GoogleLanguageModel): all 8 cells pass on gemini-3.1-flash-lite, after the review fixes and the rebase onto phases 3 and 7. I didn't run the other rows.
  • Review fixes were reproduced first and confirmed afterwards with throwaway scripts (chunked SSE for held calls, preamble text in JSON mode, text-less signatures, empty finish reasons, in-band 400/403, ID-less calls). Per the phase brief, no tests were added beyond the matrix cells.
  • Fixtures contain no stack host or token.

Deferred

  • Files inside tool results (modelOutput with files) fail with InvalidUserInputError; Foundry's support for functionResponse.parts is unverified. Text model output is sent as content.
  • Remote file URLs go out as fileData, unverified on Foundry. Base64 data URLs become inlineData.
  • Provider-defined tools (code execution, search), partialArgs streaming, cachedContent, VALIDATED mode, safety settings and CONTINUATION resubmission aren't implemented: no cell needs them.
  • System messages after the first non-system message fail with InvalidUserInputError, as in the AI SDK adapter.

@anandpant anandpant changed the title feat(effect-ai): GoogleLanguageModel over generateContent with recorded fixtures feat(effect-ai): GoogleLanguageModel over /google/v1 generateContent (phase 5) Nov 20, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-20T03:00:54.707154Z b85bb41 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b617b0b8c7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

anandpant commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor Author

Merge activity

  • Nov 20, 3:00 AM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Nov 20, 3:00 AM UTC: @anandpant merged this pull request with Graphite.

@anandpant
anandpant merged commit 9185481 into main Nov 20, 2026
8 checks passed
@anandpant
anandpant deleted the effect/phase-5 branch November 20, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant