From 0759e94205237229ef8c2eeede3343f7f483ca94 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Wed, 23 Sep 2026 15:03:20 -0400 Subject: [PATCH 01/69] Add library rule packs to amber-lsp --- docs/rule-packs.md | 101 +++++ spec/amber_lsp/analyzer_spec.cr | 8 + spec/amber_lsp/rule_packs_spec.cr | 423 ++++++++++++++++++ .../controllers/action_return_rule_spec.cr | 57 +++ spec/amber_lsp/spec_helper.cr | 8 +- src/amber_lsp.cr | 12 +- src/amber_lsp/analyzer.cr | 26 ++ src/amber_lsp/controller.cr | 6 +- .../analyze_project_files_with_rule_packs.cr | 264 +++++++++++ .../describe_library_rule_pack.cr | 119 +++++ .../determine_project_rule_pack_state.cr | 136 ++++++ .../load_rule_packs_for_project.cr | 90 ++++ .../print_declared_rule_pack_contexts.cr | 90 ++++ ...t_crystal_calls_outside_required_blocks.cr | 159 +++++++ src/amber_lsp/project_context.cr | 47 +- src/amber_lsp/rules/base_rule.cr | 4 + .../rules/controllers/action_return_rule.cr | 19 +- src/amber_lsp/rules/custom_rule.cr | 4 + src/amber_lsp/rules/rule_registry.cr | 5 + 19 files changed, 1563 insertions(+), 15 deletions(-) create mode 100644 docs/rule-packs.md create mode 100644 spec/amber_lsp/rule_packs_spec.cr create mode 100644 src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr create mode 100644 src/amber_lsp/library_rule_packs/describe_library_rule_pack.cr create mode 100644 src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr create mode 100644 src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr create mode 100644 src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts.cr create mode 100644 src/amber_lsp/library_rule_packs/visit_crystal_calls_outside_required_blocks.cr diff --git a/docs/rule-packs.md b/docs/rule-packs.md new file mode 100644 index 0000000..2a2b662 --- /dev/null +++ b/docs/rule-packs.md @@ -0,0 +1,101 @@ +# Library rule packs + +Library rule packs let a Crystal library ship opt-in feature guidance and edit-time checks with the same `.claude/` files that Shards distributes. `amber-lsp` reads packs from installed dependencies at `lib/*/.claude/rules/*.yml` and from the project at `.claude/rules/*.yml`; no copy step is required. A project pack with the same `pack` id overrides a dependency pack. A pack whose `library` equals the project's own shard name does not apply, because that pack describes the library's consumer contract. + +## Pack document + +Each YAML file has a pack identity, declared modes, optional evidence patterns, mode-specific context, and rules: + +```yaml +pack: grant/tenancy +library: grant +version: 1.0.0 + +modes: + row: + declared_by: + key_path: grant.tenancy + expected_value: row + evidence: + - '^\s*multitenant\b' + tenant_column: tenant_id + context: | + Scope queries with the library's tenant context. + schema: + declared_by: + key_path: grant.tenancy + expected_value: schema + evidence: + - '^\s*SchemaTenant\.with\b' + context: | + Schema-specific rules are not included yet. + +rules: + - id: grant/example + modes: [row] + severity: warning + applies_to: ["src/**"] + exclude_from: ["src/controllers/**"] + message: "Explain the finding and the repair." + check: + kind: line_regex + pattern: '\bunsafe_call\b' +``` + +`declared_by.key_path` is a dotted path into the application's `shard.yml`; `expected_value` is compared as a string. A pack rule runs only when one of its modes is declared. Evidence patterns scan the application's `.cr` files outside `lib/`, `.git/`, `tmp/`, and `vendor/`; they support mixed-mode detection and the undeclared-feature warning. Evidence alone does not activate mode-specific rules. Context blocks are printed only for declared modes and should stay to about 15 lines or fewer. + +Each rule requires an id, one or more mode names, a severity, file globs, a message, and one check. Severity values are `error`, `warning`, `info`, and `hint`. `exclude_from` is optional. `applies_to` and `exclude_from` use Crystal path globs, including `**` for recursive directories. Rules can be enabled, disabled, or have their severity overridden through the existing `.amber-lsp.yml` `rules` mapping. + +## Check kinds + +### `line_regex` + +Matches a regular expression on each line and uses the existing custom-rule diagnostic matcher. `negate: true` reports once at the start of a file when the pattern does not occur. + +```yaml +check: + kind: line_regex + pattern: '^\s*[^#]*\.unscoped\b' +``` + +### `file_requires` + +Reports on each line matching `trigger_pattern` when the file has no line matching `required_pattern`. If `trigger_pattern` is omitted, `amber-lsp` uses the first mode's `tenant_column` as a Crystal `column ` declaration. + +```yaml +check: + kind: file_requires + required_pattern: '^\s*multitenant\b' +``` + +### `call_outside_block` + +Uses `Crystal::Parser` and a `Crystal::Visitor` to inspect method calls. `source_globs` select the files where `tenant_macro` declarations identify scoped model names. `methods` lists calls on those model constants that can query the database. A dotted `required_call`, such as `Grant::Tenant.with`, permits calls lexically inside its block and nested blocks. `escape_call` permits calls only inside a block on that same model receiver, such as `Todo.unscoped { Todo.where(...) }`. A method definition starts a new lexical scope, so a query written in a method body does not become tenant-scoped just because the method definition appears inside a tenant block. Receiver constants are followed through chained calls, but model instances held in variables are not resolved. The current matcher uses the last constant name, so same-named models in different namespaces can be ambiguous. + +```yaml +check: + kind: call_outside_block + source_globs: ["src/models/**"] + tenant_macro: multitenant + methods: [where, find, all, first, count] + required_call: Grant::Tenant.with + escape_call: unscoped +``` + +`required_call` may be omitted when `escape_call` alone defines the allowed block. The rule intentionally does not infer receiver types or follow aliases stored in local variables. + +### `project_conflict` + +Runs over project state rather than one source line. The default `condition: mixed_modes` reports when every listed mode is declared or has evidence. `condition: evidence_without_declaration` reports when a listed mode has feature evidence but its declaration is absent; use one mode for this warning. + +```yaml +check: + kind: project_conflict + condition: mixed_modes +``` + +## Project activation and context + +Amber's built-in convention rules remain gated on an `amber` dependency. Library packs run in any project when the pack applies, including a project with only a Grant dependency. Pack diagnostics use the ordinary LSP diagnostic channel and `.amber-lsp.yml` severity overrides. + +Run `amber-lsp context [--root DIR]` to print the context blocks for declared pack modes and one warning line per pack whose feature is used without a declaration. It prints nothing when no pack is declared or evidenced. Exit status is 0 for successful context inspection; invalid command arguments or an unreadable project return nonzero. diff --git a/spec/amber_lsp/analyzer_spec.cr b/spec/amber_lsp/analyzer_spec.cr index 13f5f6e..be0d503 100644 --- a/spec/amber_lsp/analyzer_spec.cr +++ b/spec/amber_lsp/analyzer_spec.cr @@ -2,6 +2,10 @@ require "./spec_helper" # A mock rule for testing the analyzer class MockTestRule < AmberLSP::Rules::BaseRule + def requires_amber_project? : Bool + false + end + def id : String "mock/test-rule" end @@ -39,6 +43,10 @@ end # A mock rule that only applies to controller files class MockControllerRule < AmberLSP::Rules::BaseRule + def requires_amber_project? : Bool + false + end + def id : String "mock/controller-rule" end diff --git a/spec/amber_lsp/rule_packs_spec.cr b/spec/amber_lsp/rule_packs_spec.cr new file mode 100644 index 0000000..2d9f4c1 --- /dev/null +++ b/spec/amber_lsp/rule_packs_spec.cr @@ -0,0 +1,423 @@ +require "./spec_helper" +require "../../src/amber_lsp/rules/controllers/action_return_rule" + +GRANT_RULE_PACK_FIXTURE = <<-YAML + pack: grant/tenancy + library: grant + version: 1.0.0 + modes: + row: + declared_by: + key_path: grant.tenancy + expected_value: row + evidence: + - '^\\s*multitenant\\b' + tenant_column: tenant_id + context: | + Scope queries with Grant::Tenant.with. + schema: + declared_by: + key_path: grant.tenancy + expected_value: schema + evidence: + - '^\\s*Grant::SchemaTenant\\.with\\b' + - '^\\s*schema_tenant_excluded\\b' + context: | + Schema-specific rules are not included yet. + rules: + - id: grant/tenant-column-without-multitenant + modes: [row] + severity: error + applies_to: ["src/models/**"] + message: Tenant columns require multitenant. + check: + kind: file_requires + required_pattern: '^\\s*multitenant\\b' + - id: grant/tenancy-modes-mixed + modes: [row, schema] + severity: error + applies_to: ["**/*.cr"] + message: Choose one tenancy mode. + check: + kind: project_conflict + condition: mixed_modes + - id: grant/row-query-outside-tenant + modes: [row] + severity: warning + applies_to: ["**/*.cr"] + exclude_from: ["src/controllers/**"] + message: Query must be tenant-scoped. + check: + kind: call_outside_block + source_globs: ["src/models/**"] + tenant_macro: multitenant + methods: [all, where, find!] + required_call: Grant::Tenant.with + escape_call: unscoped + - id: grant/unscoped-in-request-code + modes: [row] + severity: warning + applies_to: ["src/controllers/**"] + message: Do not use unscoped in request code. + check: + kind: line_regex + pattern: '^\\s*[^#]*\\.unscoped\\b' + - id: grant/raw-sql-on-scoped-model + modes: [row] + severity: warning + applies_to: ["**/*.cr"] + message: Raw SQL must use an unscoped block. + check: + kind: call_outside_block + source_globs: ["src/models/**"] + tenant_macro: multitenant + methods: [exec, query, scalar] + escape_call: unscoped + - id: grant/tenancy-undeclared + modes: [row] + severity: warning + applies_to: ["**/*.cr"] + message: Declare Grant tenancy in shard.yml. + check: + kind: project_conflict + condition: evidence_without_declaration + YAML + +def write_rule_pack_project( + root : String, + shard_content : String = "name: tenant_app\nversion: 0.1.0\ngrant:\n tenancy: row\n", + pack_content : String = GRANT_RULE_PACK_FIXTURE, +) : Nil + Dir.mkdir_p(File.join(root, ".claude", "rules")) + File.write(File.join(root, "shard.yml"), shard_content) + File.write(File.join(root, ".claude", "rules", "tenancy.yml"), pack_content) +end + +def analyze_pack_file(root : String, file_path : String, content : String) : Array(AmberLSP::Rules::Diagnostic) + Dir.mkdir_p(File.dirname(file_path)) + File.write(file_path, content) + project_context = AmberLSP::ProjectContext.detect(root) + analyzer = AmberLSP::Analyzer.new + analyzer.configure(project_context) + analyzer.analyze(file_path, content) +end + +def diagnostic_codes(diagnostics : Array(AmberLSP::Rules::Diagnostic)) : Array(String) + diagnostics.map(&.code) +end + +describe "AmberLSP library rule packs" do + before_each do + AmberLSP::Rules::RuleRegistry.clear + end + + it "loads packs from installed dependencies and project rules" do + with_tempdir do |root| + write_rule_pack_project(root) + dependency_root = File.join(root, "dependency_source") + dependency_pack_path = File.join(dependency_root, ".claude", "rules", "tenancy.yml") + project_pack_path = File.join(root, ".claude", "rules", "project.yml") + Dir.mkdir_p(File.dirname(dependency_pack_path)) + File.write(dependency_pack_path, GRANT_RULE_PACK_FIXTURE) + Dir.mkdir_p(File.join(root, "lib")) + File.symlink(dependency_root, File.join(root, "lib", "grant")) + File.write( + project_pack_path, + GRANT_RULE_PACK_FIXTURE.gsub("grant/tenancy", "project/tenancy").gsub("library: grant", "library: project"), + ) + + project_context = AmberLSP::ProjectContext.detect(root) + packs = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs + + packs.map(&.pack_id).should eq(["grant/tenancy", "project/tenancy"]) + end + end + + it "keeps a library's own pack inactive while allowing clean LSP checks" do + with_tempdir do |root| + shard_content = "name: grant\nversion: 0.1.0\n" + write_rule_pack_project(root, shard_content) + + project_context = AmberLSP::ProjectContext.detect(root) + packs = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs + packs.map(&.pack_id).should eq(["grant/tenancy"]) + + file_path = File.join(root, "src", "grant", "scale", "tenant.cr") + content = "multitenant :tenant_id\n" + analyzer = AmberLSP::Analyzer.new + analyzer.configure(project_context) + analyzer.has_applicable_library_rule_pack?(file_path, content).should be_true + analyzer.analyze(file_path, content).should be_empty + end + end + + it "runs a declared dependency pack in a non-Amber project" do + with_tempdir do |root| + write_rule_pack_project(root) + Dir.mkdir_p(File.join(root, "src", "jobs")) + file_path = File.join(root, "src", "jobs", "fixture.cr") + diagnostics = analyze_pack_file(root, file_path, "puts \"unscoped\"\n") + + diagnostics.map(&.code).should eq([] of String) + end + end + + it "runs line_regex checks and ignores full-line comments" do + with_tempdir do |root| + write_rule_pack_project(root) + file_path = File.join(root, "src", "controllers", "todos_controller.cr") + content = "# Todo.unscoped is only documentation\nTodo.unscoped.all\n" + + diagnostics = analyze_pack_file(root, file_path, content) + + diagnostics.map(&.code).should eq(["grant/unscoped-in-request-code"]) + diagnostics.first.range.start.line.should eq(1) + diagnostics.first.severity.should eq(AmberLSP::Rules::Severity::Warning) + end + end + + it "reports a tenant column without multitenant and accepts the declared macro" do + with_tempdir do |root| + write_rule_pack_project(root) + file_path = File.join(root, "src", "models", "account.cr") + Dir.mkdir_p(File.dirname(file_path)) + missing_macro = "class Account\n column tenant_id : Int64\nend\n" + + diagnostics = analyze_pack_file(root, file_path, missing_macro) + + diagnostics.map(&.code).should contain("grant/tenant-column-without-multitenant") + diagnostics.find(&.code.==("grant/tenant-column-without-multitenant")).not_nil!.severity.should eq(AmberLSP::Rules::Severity::Error) + + valid_model = "class Account\n column tenant_id : Int64\n multitenant :tenant_id\nend\n" + valid_diagnostics = analyze_pack_file(root, file_path, valid_model) + + valid_diagnostics.map(&.code).should_not contain("grant/tenant-column-without-multitenant") + end + end + + it "uses the configured tenant column for file_requires" do + with_tempdir do |root| + pack_content = GRANT_RULE_PACK_FIXTURE.gsub("tenant_column: tenant_id", "tenant_column: account_id") + write_rule_pack_project(root, pack_content: pack_content) + file_path = File.join(root, "src", "models", "account.cr") + Dir.mkdir_p(File.dirname(file_path)) + + diagnostics = analyze_pack_file(root, file_path, "class Account\n column account_id : Int64\nend\n") + + diagnostics.map(&.code).should contain("grant/tenant-column-without-multitenant") + end + end + + it "reports a multitenant query outside the required block" do + with_tempdir do |root| + write_rule_pack_project(root) + Dir.mkdir_p(File.join(root, "src", "models")) + File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") + file_path = File.join(root, "src", "jobs", "cleanup_job.cr") + Dir.mkdir_p(File.dirname(file_path)) + + diagnostics = analyze_pack_file(root, file_path, "Todo.where(active: true)\n") + + diagnostics.map(&.code).should contain("grant/row-query-outside-tenant") + end + end + + it "accepts queries inside the required block and nested blocks" do + with_tempdir do |root| + write_rule_pack_project(root) + Dir.mkdir_p(File.join(root, "src", "models")) + File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") + file_path = File.join(root, "src", "jobs", "cleanup_job.cr") + Dir.mkdir_p(File.dirname(file_path)) + content = <<-CRYSTAL + Grant::Tenant.with(7) do + Todo.where(active: true) + run do + Todo.all + end + end + CRYSTAL + + diagnostics = analyze_pack_file(root, file_path, content) + + diagnostics.map(&.code).should_not contain("grant/row-query-outside-tenant") + end + end + + it "does not treat a method defined inside a tenant block as scoped" do + with_tempdir do |root| + write_rule_pack_project(root) + Dir.mkdir_p(File.join(root, "src", "models")) + File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") + file_path = File.join(root, "src", "jobs", "cleanup_job.cr") + Dir.mkdir_p(File.dirname(file_path)) + content = <<-CRYSTAL + Grant::Tenant.with(7) do + def load_todos + Todo.all + end + end + CRYSTAL + + diagnostics = analyze_pack_file(root, file_path, content) + + diagnostics.map(&.code).should contain("grant/row-query-outside-tenant") + end + end + + it "does not scope a method body just because its call is inside a tenant block" do + with_tempdir do |root| + write_rule_pack_project(root) + Dir.mkdir_p(File.join(root, "src", "models")) + File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") + file_path = File.join(root, "src", "jobs", "cleanup_job.cr") + Dir.mkdir_p(File.dirname(file_path)) + content = <<-CRYSTAL + def load_todos + Todo.all + end + Grant::Tenant.with(7) { load_todos } + CRYSTAL + + diagnostics = analyze_pack_file(root, file_path, content) + + diagnostics.map(&.code).should contain("grant/row-query-outside-tenant") + end + end + + it "allows a same-model unscoped block but not a different model's query" do + with_tempdir do |root| + write_rule_pack_project(root) + Dir.mkdir_p(File.join(root, "src", "models")) + File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") + File.write(File.join(root, "src", "models", "invoice.cr"), "class Invoice\n multitenant :tenant_id\nend\n") + file_path = File.join(root, "src", "jobs", "cleanup_job.cr") + Dir.mkdir_p(File.dirname(file_path)) + + same_model = analyze_pack_file(root, file_path, "Todo.unscoped { Todo.where(active: true) }\n") + same_model.map(&.code).should_not contain("grant/row-query-outside-tenant") + + different_model = analyze_pack_file(root, file_path, "Todo.unscoped { Invoice.all }\n") + different_model.map(&.code).should contain("grant/row-query-outside-tenant") + end + end + + it "does not flag raw_all and allows raw SQL inside the same model's unscoped block" do + with_tempdir do |root| + write_rule_pack_project(root) + Dir.mkdir_p(File.join(root, "src", "models")) + File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") + file_path = File.join(root, "src", "jobs", "cleanup_job.cr") + Dir.mkdir_p(File.dirname(file_path)) + + raw_all_diagnostics = analyze_pack_file(root, file_path, "Todo.raw_all(\"WHERE active = true\")\n") + raw_all_diagnostics.map(&.code).should_not contain("grant/raw-sql-on-scoped-model") + + scoped_sql = <<-CRYSTAL + Todo.unscoped do + Todo.exec("DELETE FROM todos") + Todo.query("SELECT 1") { } + Todo.scalar("SELECT COUNT(*) FROM todos") { |value| value } + end + CRYSTAL + scoped_sql_diagnostics = analyze_pack_file(root, file_path, scoped_sql) + scoped_sql_diagnostics.map(&.code).should_not contain("grant/raw-sql-on-scoped-model") + + unsafe_sql = "Todo.exec(\"DELETE FROM todos\")\nTodo.query(\"SELECT 1\") { }\nTodo.scalar(\"SELECT 1\") { |value| value }\n" + unsafe_sql_diagnostics = analyze_pack_file(root, file_path, unsafe_sql) + unsafe_sql_diagnostics.count(&.code.==("grant/raw-sql-on-scoped-model")).should eq(3) + end + end + + it "reports mixed modes as an error and undeclared row use as a warning" do + with_tempdir do |root| + write_rule_pack_project(root) + file_path = File.join(root, "src", "jobs", "tenancy_job.cr") + Dir.mkdir_p(File.dirname(file_path)) + mixed_content = "Grant::SchemaTenant.with(\"acme\") { run_job }\n" + + mixed_diagnostics = analyze_pack_file(root, file_path, mixed_content) + mixed_diagnostic = mixed_diagnostics.find(&.code.==("grant/tenancy-modes-mixed")).not_nil! + mixed_diagnostic.severity.should eq(AmberLSP::Rules::Severity::Error) + + shard_content = "name: tenant_app\nversion: 0.1.0\n" + write_rule_pack_project(root, shard_content) + Dir.mkdir_p(File.join(root, "src", "models")) + File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") + undeclared_path = File.join(root, "src", "jobs", "undeclared_job.cr") + undeclared_diagnostics = analyze_pack_file(root, undeclared_path, "Todo.all\n") + undeclared = undeclared_diagnostics.find(&.code.==("grant/tenancy-undeclared")).not_nil! + undeclared.severity.should eq(AmberLSP::Rules::Severity::Warning) + end + end + + it "keeps Amber built-in rules gated on Amber dependencies" do + with_tempdir do |root| + write_rule_pack_project(root) + file_path = File.join(root, "src", "controllers", "home_controller.cr") + Dir.mkdir_p(File.dirname(file_path)) + content = "class HomeController < ApplicationController\n def index\n User.all\n end\nend\n" + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::Controllers::ActionReturnRule.new) + non_amber_analyzer = AmberLSP::Analyzer.new + non_amber_analyzer.configure(AmberLSP::ProjectContext.detect(root)) + + non_amber_diagnostics = non_amber_analyzer.analyze(file_path, content) + non_amber_diagnostics.map(&.code).should_not contain("amber/action-return-type") + + shard_content = <<-YAML + name: tenant_app + version: 0.1.0 + grant: + tenancy: row + dependencies: + amber: + github: amberframework/amber + YAML + File.write(File.join(root, "shard.yml"), shard_content) + amber_analyzer = AmberLSP::Analyzer.new + amber_analyzer.configure(AmberLSP::ProjectContext.detect(root)) + + amber_diagnostics = amber_analyzer.analyze(file_path, content) + amber_diagnostics.map(&.code).should contain("amber/action-return-type") + end + end + + it "prints declared context and warns when feature use has no declaration" do + with_tempdir do |root| + write_rule_pack_project(root) + file_path = File.join(root, "src", "models", "todo.cr") + Dir.mkdir_p(File.dirname(file_path)) + File.write(file_path, "class Todo\n multitenant :tenant_id\nend\n") + binary_path = File.join(Dir.current, "bin", "amber-lsp") + stdout = IO::Memory.new + stderr = IO::Memory.new + status = Process.run(binary_path, ["context", "--root", root], output: stdout, error: stderr) + + status.success?.should be_true + stderr.to_s.should be_empty + stdout.to_s.should contain("grant/tenancy (row)") + stdout.to_s.should contain("Scope queries with Grant::Tenant.with.") + + undeclared_shard = "name: tenant_app\nversion: 0.1.0\n" + write_rule_pack_project(root, undeclared_shard) + stdout = IO::Memory.new + stderr = IO::Memory.new + status = Process.run(binary_path, ["context", "--root", root], output: stdout, error: stderr) + + status.success?.should be_true + stdout.to_s.should contain("warning: grant/tenancy feature is used but shard.yml does not declare grant.tenancy.") + end + end + + it "prints nothing when no pack is declared or evidenced" do + with_tempdir do |root| + File.write(File.join(root, "shard.yml"), "name: empty_app\nversion: 0.1.0\n") + binary_path = File.join(Dir.current, "bin", "amber-lsp") + stdout = IO::Memory.new + status = Process.run(binary_path, ["context", "--root", root], output: stdout, error: Process::Redirect::Close) + + status.success?.should be_true + stdout.to_s.should be_empty + end + end +end diff --git a/spec/amber_lsp/rules/controllers/action_return_rule_spec.cr b/spec/amber_lsp/rules/controllers/action_return_rule_spec.cr index 7a0b6fb..76ff0ee 100644 --- a/spec/amber_lsp/rules/controllers/action_return_rule_spec.cr +++ b/spec/amber_lsp/rules/controllers/action_return_rule_spec.cr @@ -80,6 +80,63 @@ describe AmberLSP::Rules::Controllers::ActionReturnRule do diagnostics.should be_empty end + it "produces no diagnostics when actions use application response helpers" do + content = <<-CRYSTAL + class HomeController < ApplicationController + def index + respond_json(200, {ok: true}) + end + end + CRYSTAL + + rule = AmberLSP::Rules::Controllers::ActionReturnRule.new + diagnostics = rule.check("src/controllers/home_controller.cr", content) + diagnostics.should be_empty + end + + it "produces no diagnostics when actions write the response directly" do + content = <<-CRYSTAL + class HomeController < ApplicationController + def index + context.response.print("ok") + end + end + CRYSTAL + + rule = AmberLSP::Rules::Controllers::ActionReturnRule.new + diagnostics = rule.check("src/controllers/home_controller.cr", content) + diagnostics.should be_empty + end + + it "produces no diagnostics when actions hand the response to a writer helper" do + content = <<-CRYSTAL + class HomeController < ApplicationController + def connect + send_event(context.response, "endpoint") + end + end + CRYSTAL + + rule = AmberLSP::Rules::Controllers::ActionReturnRule.new + diagnostics = rule.check("src/controllers/home_controller.cr", content) + diagnostics.should be_empty + end + + it "skips class methods and controller support files" do + content = <<-CRYSTAL + class HomeController < ApplicationController + def self.build + "helper" + end + end + CRYSTAL + + rule = AmberLSP::Rules::Controllers::ActionReturnRule.new + rule.check("src/controllers/home_controller.cr", content).should be_empty + rule.check("src/controllers/application_controller.cr", content).should be_empty + rule.check("src/controllers/concerns/session_state.cr", content).should be_empty + end + it "reports warning when action does not call any response method" do content = <<-CRYSTAL class HomeController < ApplicationController diff --git a/spec/amber_lsp/spec_helper.cr b/spec/amber_lsp/spec_helper.cr index 93c6d9f..822b2f0 100644 --- a/spec/amber_lsp/spec_helper.cr +++ b/spec/amber_lsp/spec_helper.cr @@ -9,6 +9,12 @@ require "../../src/amber_lsp/rules/custom_rule" require "../../src/amber_lsp/document_store" require "../../src/amber_lsp/project_context" require "../../src/amber_lsp/configuration" +require "../../src/amber_lsp/library_rule_packs/describe_library_rule_pack" +require "../../src/amber_lsp/library_rule_packs/load_rule_packs_for_project" +require "../../src/amber_lsp/library_rule_packs/visit_crystal_calls_outside_required_blocks" +require "../../src/amber_lsp/library_rule_packs/determine_project_rule_pack_state" +require "../../src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs" +require "../../src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts" require "../../src/amber_lsp/analyzer" require "../../src/amber_lsp/controller" require "../../src/amber_lsp/server" @@ -29,7 +35,7 @@ def format_lsp_message(message) : String end def run_lsp_session(messages : Array) : Array(JSON::Any) - input_data = messages.map { |m| format_lsp_message(m) }.join + input_data = messages.map { |message| format_lsp_message(message) }.join input = IO::Memory.new(input_data) output = IO::Memory.new diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 472f055..d4ae72e 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -21,8 +21,18 @@ require "./amber_lsp/rules/custom_rule" require "./amber_lsp/document_store" require "./amber_lsp/project_context" require "./amber_lsp/configuration" +require "./amber_lsp/library_rule_packs/describe_library_rule_pack" +require "./amber_lsp/library_rule_packs/load_rule_packs_for_project" +require "./amber_lsp/library_rule_packs/visit_crystal_calls_outside_required_blocks" +require "./amber_lsp/library_rule_packs/determine_project_rule_pack_state" +require "./amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs" +require "./amber_lsp/library_rule_packs/print_declared_rule_pack_contexts" require "./amber_lsp/analyzer" require "./amber_lsp/controller" require "./amber_lsp/server" -AmberLSP::Server.new(STDIN, STDOUT).run +if ARGV.first? == "context" + exit AmberLSP::LibraryRulePacks::PrintDeclaredRulePackContexts.new(ARGV[1..].to_a).perform +else + AmberLSP::Server.new(STDIN, STDOUT).run +end diff --git a/src/amber_lsp/analyzer.cr b/src/amber_lsp/analyzer.cr index 58186a4..30c8140 100644 --- a/src/amber_lsp/analyzer.cr +++ b/src/amber_lsp/analyzer.cr @@ -2,18 +2,36 @@ module AmberLSP class Analyzer getter configuration : Configuration @project_root : String? + @project_context : ProjectContext? + @library_rule_pack_analyzer : LibraryRulePacks::AnalyzeProjectFilesWithRulePacks? def initialize @configuration = Configuration.new @project_root = nil + @project_context = nil + @library_rule_pack_analyzer = nil end def configure(project_context : ProjectContext) : Nil @configuration = Configuration.load(project_context.root_path) @project_root = project_context.root_path + @project_context = project_context + list_of_rule_packs = LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs + @library_rule_pack_analyzer = LibraryRulePacks::AnalyzeProjectFilesWithRulePacks.new( + project_context, + @configuration, + list_of_rule_packs, + ) register_custom_rules end + def has_applicable_library_rule_pack?(file_path : String, content : String) : Bool + analyzer = @library_rule_pack_analyzer + return false unless analyzer + + analyzer.has_applicable_pack?(file_path, content) + end + private def register_custom_rules : Nil @configuration.custom_rules.each do |custom_config| severity = case custom_config.severity @@ -48,6 +66,10 @@ module AmberLSP rules.each do |rule| next unless @configuration.rule_enabled?(rule.id) + if rule.requires_amber_project? + project_context = @project_context + next unless project_context && project_context.amber_project? + end rule_diagnostics = rule.check(file_path, content) severity = @configuration.rule_severity(rule.id, rule.default_severity) @@ -67,6 +89,10 @@ module AmberLSP end end + if analyzer = @library_rule_pack_analyzer + diagnostics.concat(analyzer.list_of_diagnostics_for(file_path, content)) + end + diagnostics end diff --git a/src/amber_lsp/controller.cr b/src/amber_lsp/controller.cr index 328ead6..61204cb 100644 --- a/src/amber_lsp/controller.cr +++ b/src/amber_lsp/controller.cr @@ -168,9 +168,9 @@ module AmberLSP # Only analyze Crystal files return unless file_path.ends_with?(".cr") - # Only run if we detected an Amber project ctx = @project_context - return unless ctx && ctx.amber_project? + return unless ctx + return unless ctx.amber_project? || @analyzer.has_applicable_library_rule_pack?(file_path, content) diagnostics = @analyzer.analyze(file_path, content) publish_diagnostics(uri, diagnostics, server) @@ -184,7 +184,7 @@ module AmberLSP "method" => JSON::Any.new("textDocument/publishDiagnostics"), "params" => JSON::Any.new({ "uri" => JSON::Any.new(uri), - "diagnostics" => JSON::Any.new(lsp_diagnostics.map { |d| JSON::Any.new(d) }), + "diagnostics" => JSON::Any.new(lsp_diagnostics.map { |diagnostic| JSON::Any.new(diagnostic) }), }), } diff --git a/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr b/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr new file mode 100644 index 0000000..da0595d --- /dev/null +++ b/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr @@ -0,0 +1,264 @@ +module AmberLSP::LibraryRulePacks + class AnalyzeProjectFilesWithRulePacks + def initialize( + @project_context : AmberLSP::ProjectContext, + @configuration : AmberLSP::Configuration, + @list_of_rule_packs : Array(DescribeLibraryRulePack), + ) + end + + def has_applicable_pack?(file_path : String, content : String) : Bool + @list_of_rule_packs.any? do |rule_pack| + next true if library_is_project_root?(rule_pack) + + DetermineProjectRulePackState.new(@project_context, rule_pack, file_path, content) + .has_any_applicable_mode? + end + end + + def list_of_diagnostics_for(file_path : String, content : String) : Array(Rules::Diagnostic) + relative_file_path = project_relative_path(file_path) + list_of_diagnostics = [] of Rules::Diagnostic + + @list_of_rule_packs.each do |rule_pack| + next if library_is_project_root?(rule_pack) + + project_state = DetermineProjectRulePackState.new( + @project_context, + rule_pack, + file_path, + content, + ) + + rule_pack.list_of_rules.each do |rule| + next unless @configuration.rule_enabled?(rule.rule_id) + next unless rule_applies_to_file?(rule, relative_file_path) + + if rule.check.check_kind == "project_conflict" + list_of_diagnostics.concat( + project_conflict_diagnostics_for(rule_pack, rule, project_state) + ) + next + end + + next unless project_state.is_rule_mode_declared?(rule.list_of_mode_names) + + list_of_diagnostics.concat( + file_diagnostics_for(rule_pack, rule, project_state, file_path, relative_file_path, content) + ) + end + end + + apply_configured_severity(list_of_diagnostics) + end + + private def library_is_project_root?(rule_pack : DescribeLibraryRulePack) : Bool + @project_context.shard_name == rule_pack.library_shard_name + end + + private def rule_applies_to_file?(rule : DescribeLibraryRulePack::Rule, relative_file_path : String) : Bool + applies = rule.list_of_applicable_globs.any? do |pattern| + Rules::RuleRegistry.file_matches_pattern?(relative_file_path, pattern) + end + return false unless applies + + rule.list_of_excluded_globs.none? do |pattern| + Rules::RuleRegistry.file_matches_pattern?(relative_file_path, pattern) + end + end + + private def project_conflict_diagnostics_for( + rule_pack : DescribeLibraryRulePack, + rule : DescribeLibraryRulePack::Rule, + project_state : DetermineProjectRulePackState, + ) : Array(Rules::Diagnostic) + conflict_found = case rule.check.project_condition + when "mixed_modes" + project_state.all_modes_are_present?(rule.list_of_mode_names) + when "evidence_without_declaration" + project_state.has_undeclared_mode_evidence?(rule.list_of_mode_names) + else + false + end + return [] of Rules::Diagnostic unless conflict_found + + [diagnostic_at_start_of_file(rule, severity_for(rule.severity_name))] + end + + private def file_diagnostics_for( + rule_pack : DescribeLibraryRulePack, + rule : DescribeLibraryRulePack::Rule, + project_state : DetermineProjectRulePackState, + file_path : String, + relative_file_path : String, + content : String, + ) : Array(Rules::Diagnostic) + case rule.check.check_kind + when "line_regex" + line_regex_diagnostics_for(rule, relative_file_path, content) + when "file_requires" + file_requires_diagnostics_for(rule_pack, rule, content) + when "call_outside_block" + call_outside_block_diagnostics_for(rule, project_state, file_path, content) + else + [] of Rules::Diagnostic + end + end + + private def line_regex_diagnostics_for( + rule : DescribeLibraryRulePack::Rule, + relative_file_path : String, + content : String, + ) : Array(Rules::Diagnostic) + custom_rule = Rules::CustomRule.new( + id: rule.rule_id, + description: rule.diagnostic_message, + default_severity: severity_for(rule.severity_name), + applies_to: rule.list_of_applicable_globs, + pattern: Regex.new(rule.check.regex_pattern), + message_template: rule.diagnostic_message, + negate: rule.check.negates_pattern?, + ) + custom_rule.check(relative_file_path, content) + end + + private def file_requires_diagnostics_for( + rule_pack : DescribeLibraryRulePack, + rule : DescribeLibraryRulePack::Rule, + content : String, + ) : Array(Rules::Diagnostic) + required_pattern = Regex.new(rule.check.required_regex_pattern) + return [] of Rules::Diagnostic if content.each_line.any? { |line| required_pattern.matches?(line) } + + trigger_pattern = trigger_pattern_for(rule_pack, rule) + return [] of Rules::Diagnostic unless trigger_pattern + + list_of_diagnostics = [] of Rules::Diagnostic + content.each_line.with_index do |line, line_number| + match = trigger_pattern.match(line) + next unless match + + start_character = (match.begin(0) || 0).to_i32 + end_character = (match.end(0) || line.size).to_i32 + range = Rules::TextRange.new( + Rules::Position.new(line_number.to_i32, start_character), + Rules::Position.new(line_number.to_i32, end_character), + ) + list_of_diagnostics << Rules::Diagnostic.new( + range, + severity_for(rule.severity_name), + rule.rule_id, + rule.diagnostic_message, + ) + end + + list_of_diagnostics + end + + private def trigger_pattern_for( + rule_pack : DescribeLibraryRulePack, + rule : DescribeLibraryRulePack::Rule, + ) : Regex? + unless rule.check.trigger_regex_pattern.empty? + return Regex.new(rule.check.trigger_regex_pattern) + end + + mode = rule_pack.modes_by_name[rule.list_of_mode_names.first]? + tenant_column_name = mode.try(&.tenant_column_name) + return nil unless tenant_column_name + return nil unless tenant_column_name.matches?(/\A[a-zA-Z_][a-zA-Z0-9_]*\z/) + + Regex.new("^\\s*column\\s+#{tenant_column_name}\\b") + end + + private def call_outside_block_diagnostics_for( + rule : DescribeLibraryRulePack::Rule, + project_state : DetermineProjectRulePackState, + file_path : String, + content : String, + ) : Array(Rules::Diagnostic) + check = rule.check + list_of_scoped_model_names = project_state.list_of_scoped_model_names_for( + check.list_of_source_globs, + check.model_macro_name, + ) + return [] of Rules::Diagnostic if list_of_scoped_model_names.empty? + + visitor = VisitCrystalCallsOutsideRequiredBlocks.new( + list_of_scoped_model_names, + check.list_of_query_method_names, + check.required_block_call_name, + check.escape_block_call_name, + check.model_macro_name, + ) + ast = Crystal::Parser.new(content).parse + visitor.accept(ast) + + visitor.list_of_calls_outside_required_blocks.compact_map do |call| + location = call.name_location || call.location + next unless location + + line = (location.line_number - 1).to_i32 + start_character = (location.column_number - 1).to_i32 + range = Rules::TextRange.new( + Rules::Position.new(line, start_character), + Rules::Position.new(line, start_character + call.name.size), + ) + Rules::Diagnostic.new( + range, + severity_for(rule.severity_name), + rule.rule_id, + rule.diagnostic_message, + ) + end + rescue Crystal::SyntaxException + [] of Rules::Diagnostic + end + + private def diagnostic_at_start_of_file( + rule : DescribeLibraryRulePack::Rule, + severity : Rules::Severity, + ) : Rules::Diagnostic + range = Rules::TextRange.new( + Rules::Position.new(0, 0), + Rules::Position.new(0, 0), + ) + Rules::Diagnostic.new(range, severity, rule.rule_id, rule.diagnostic_message) + end + + private def apply_configured_severity( + list_of_diagnostics : Array(Rules::Diagnostic), + ) : Array(Rules::Diagnostic) + list_of_diagnostics.map do |diagnostic| + configured_severity = @configuration.rule_severity(diagnostic.code, diagnostic.severity) + next diagnostic if configured_severity == diagnostic.severity + + Rules::Diagnostic.new( + diagnostic.range, + configured_severity, + diagnostic.code, + diagnostic.message, + diagnostic.source, + ) + end + end + + private def severity_for(severity_name : String) : Rules::Severity + case severity_name.downcase + when "error" then Rules::Severity::Error + when "info" then Rules::Severity::Information + when "hint" then Rules::Severity::Hint + else Rules::Severity::Warning + end + end + + private def project_relative_path(file_path : String) : String + project_root = File.expand_path(@project_context.root_path) + absolute_file_path = File.expand_path(file_path) + prefix = project_root.ends_with?(File::SEPARATOR) ? project_root : "#{project_root}#{File::SEPARATOR}" + return file_path unless absolute_file_path.starts_with?(prefix) + + absolute_file_path[prefix.size..] + end + end +end diff --git a/src/amber_lsp/library_rule_packs/describe_library_rule_pack.cr b/src/amber_lsp/library_rule_packs/describe_library_rule_pack.cr new file mode 100644 index 0000000..7d1a550 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/describe_library_rule_pack.cr @@ -0,0 +1,119 @@ +require "yaml" + +module AmberLSP::LibraryRulePacks + class DescribeLibraryRulePack + include YAML::Serializable + + @[YAML::Field(key: "pack")] + property pack_id : String + + @[YAML::Field(key: "library")] + property library_shard_name : String + + @[YAML::Field(key: "version")] + property pack_version : String + + @[YAML::Field(key: "modes")] + property modes_by_name : Hash(String, Mode) = {} of String => Mode + + @[YAML::Field(key: "rules")] + property list_of_rules : Array(Rule) = [] of Rule + + def is_valid? : Bool + return false if @pack_id.empty? || @library_shard_name.empty? || @pack_version.empty? + return false if @modes_by_name.empty? + + @list_of_rules.all? do |rule| + !rule.rule_id.empty? && + !rule.list_of_mode_names.empty? && + rule.list_of_mode_names.all? { |mode_name| @modes_by_name.has_key?(mode_name) } && + !rule.list_of_applicable_globs.empty? && + !rule.check.check_kind.empty? + end + end + + class Declaration + include YAML::Serializable + + property key_path : String + property expected_value : String + end + + class Mode + include YAML::Serializable + + @[YAML::Field(key: "declared_by")] + property declaration : Declaration + + @[YAML::Field(key: "evidence")] + property list_of_evidence_patterns : Array(String) = [] of String + + @[YAML::Field(key: "tenant_column")] + property tenant_column_name : String? = nil + + @[YAML::Field(key: "context")] + property guidance_text : String = "" + end + + class Rule + include YAML::Serializable + + @[YAML::Field(key: "id")] + property rule_id : String + + @[YAML::Field(key: "modes")] + property list_of_mode_names : Array(String) = [] of String + + @[YAML::Field(key: "severity")] + property severity_name : String = "warning" + + @[YAML::Field(key: "applies_to")] + property list_of_applicable_globs : Array(String) = ["**/*.cr"] + + @[YAML::Field(key: "exclude_from")] + property list_of_excluded_globs : Array(String) = [] of String + + @[YAML::Field(key: "message")] + property diagnostic_message : String = "" + + property check : Check + end + + class Check + include YAML::Serializable + + @[YAML::Field(key: "kind")] + property check_kind : String + + @[YAML::Field(key: "pattern")] + property regex_pattern : String = "" + + @[YAML::Field(key: "trigger_pattern")] + property trigger_regex_pattern : String = "" + + @[YAML::Field(key: "required_pattern")] + property required_regex_pattern : String = "" + + @[YAML::Field(key: "negate")] + property? negates_pattern : Bool = false + + @[YAML::Field(key: "source_globs")] + property list_of_source_globs : Array(String) = [] of String + + @[YAML::Field(key: "tenant_macro")] + property model_macro_name : String = "multitenant" + + @[YAML::Field(key: "methods")] + property list_of_query_method_names : Array(String) = [] of String + + @[YAML::Field(key: "required_call")] + property required_block_call_name : String = "" + + @[YAML::Field(key: "escape_call")] + property escape_block_call_name : String = "" + + @[YAML::Field(key: "condition")] + property project_condition : String = "mixed_modes" + end + end +end diff --git a/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr b/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr new file mode 100644 index 0000000..dd7ccc8 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr @@ -0,0 +1,136 @@ +require "set" + +module AmberLSP::LibraryRulePacks + class DetermineProjectRulePackState + @is_mode_declared_by_name = {} of String => Bool + @has_mode_evidence_by_name = {} of String => Bool + @project_source_content_by_path = {} of String => String + + def initialize( + @project_context : AmberLSP::ProjectContext, + @rule_pack : DescribeLibraryRulePack, + @file_path : String, + @file_content : String, + ) + load_project_source_content + determine_mode_declarations + determine_mode_evidence + end + + def is_mode_declared?(mode_name : String) : Bool + @is_mode_declared_by_name[mode_name]? || false + end + + def has_evidence_for_mode?(mode_name : String) : Bool + @has_mode_evidence_by_name[mode_name]? || false + end + + def is_mode_present?(mode_name : String) : Bool + is_mode_declared?(mode_name) || has_evidence_for_mode?(mode_name) + end + + def has_any_applicable_mode? : Bool + @rule_pack.modes_by_name.keys.any? do |mode_name| + is_mode_declared?(mode_name) || has_evidence_for_mode?(mode_name) + end + end + + def all_modes_are_present?(list_of_mode_names : Array(String)) : Bool + !list_of_mode_names.empty? && list_of_mode_names.all? { |mode_name| is_mode_present?(mode_name) } + end + + def has_undeclared_mode_evidence?(list_of_mode_names : Array(String)) : Bool + list_of_mode_names.any? do |mode_name| + has_evidence_for_mode?(mode_name) && !is_mode_declared?(mode_name) + end + end + + def is_rule_mode_declared?(list_of_mode_names : Array(String)) : Bool + list_of_mode_names.any? { |mode_name| is_mode_declared?(mode_name) } + end + + def list_of_scoped_model_names_for( + source_globs : Array(String), + model_macro_name : String, + ) : Set(String) + list_of_model_names = Set(String).new + + @project_source_content_by_path.each do |file_path, content| + relative_path = project_relative_path(file_path) + next unless source_globs.any? { |pattern| Rules::RuleRegistry.file_matches_pattern?(relative_path, pattern) } + + list_of_model_names.concat( + VisitCrystalCallsOutsideRequiredBlocks.find_multitenant_model_names(content, model_macro_name) + ) + end + + list_of_model_names + end + + private def load_project_source_content : Nil + project_root = File.expand_path(@project_context.root_path) + + Dir.glob(File.join(project_root, "**", "*.cr")).each do |file_path| + next unless project_file_is_application_source?(file_path, project_root) + + content = if File.expand_path(file_path) == File.expand_path(@file_path) + @file_content + else + File.read(file_path) + end + @project_source_content_by_path[File.expand_path(file_path)] = content + rescue + next + end + + current_file_path = File.expand_path(@file_path) + if project_file_is_application_source?(current_file_path, project_root) + @project_source_content_by_path[current_file_path] = @file_content + end + end + + private def project_file_is_application_source?(file_path : String, project_root : String) : Bool + prefix = project_root.ends_with?(File::SEPARATOR) ? project_root : "#{project_root}#{File::SEPARATOR}" + return false unless file_path.starts_with?(prefix) + + relative_path = file_path[prefix.size..] + return false if relative_path.starts_with?("lib/") + return false if relative_path.starts_with?(".git/") + return false if relative_path.starts_with?("tmp/") + return false if relative_path.starts_with?("vendor/") + + true + end + + private def determine_mode_declarations : Nil + @rule_pack.modes_by_name.each do |mode_name, mode| + @is_mode_declared_by_name[mode_name] = @project_context.has_shard_declaration?( + mode.declaration.key_path, + mode.declaration.expected_value, + ) + end + end + + private def determine_mode_evidence : Nil + @rule_pack.modes_by_name.each do |mode_name, mode| + evidence_found = mode.list_of_evidence_patterns.any? do |pattern| + regex = Regex.new(pattern) + @project_source_content_by_path.values.any? do |content| + content.each_line.any? { |line| regex.matches?(line) } + end + end + @has_mode_evidence_by_name[mode_name] = evidence_found + end + rescue ArgumentError + @rule_pack.modes_by_name.each_key { |mode_name| @has_mode_evidence_by_name[mode_name] = false } + end + + private def project_relative_path(file_path : String) : String + project_root = File.expand_path(@project_context.root_path) + prefix = project_root.ends_with?(File::SEPARATOR) ? project_root : "#{project_root}#{File::SEPARATOR}" + return file_path unless file_path.starts_with?(prefix) + + file_path[prefix.size..] + end + end +end diff --git a/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr b/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr new file mode 100644 index 0000000..1becfd1 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr @@ -0,0 +1,90 @@ +module AmberLSP::LibraryRulePacks + class LoadRulePacksForProject + def initialize(@project_context : AmberLSP::ProjectContext) + end + + def load_rule_packs : Array(DescribeLibraryRulePack) + list_of_packs_by_id = {} of String => DescribeLibraryRulePack + + dependency_pack_paths.each do |pack_path| + load_rule_pack(pack_path).try do |rule_pack| + list_of_packs_by_id[rule_pack.pack_id] = rule_pack + end + end + + project_pack_paths.each do |pack_path| + load_rule_pack(pack_path).try do |rule_pack| + list_of_packs_by_id[rule_pack.pack_id] = rule_pack + end + end + + list_of_packs_by_id.values.sort_by(&.pack_id) + end + + private def dependency_pack_paths : Array(String) + list_of_library_paths = Dir.glob(File.join(@project_context.root_path, "lib", "*")).sort + list_of_library_paths.flat_map do |library_path| + Dir.glob(File.join(library_path, ".claude", "rules", "*.yml")) + end.sort + end + + private def project_pack_paths : Array(String) + Dir.glob(File.join(@project_context.root_path, ".claude", "rules", "*.yml")).sort + end + + private def load_rule_pack(pack_path : String) : DescribeLibraryRulePack? + rule_pack = DescribeLibraryRulePack.from_yaml(File.read(pack_path)) + return rule_pack if rule_pack_is_valid?(rule_pack) + + STDERR.puts "WARNING: Ignoring invalid amber-lsp rule pack at #{pack_path}." + nil + rescue ex + STDERR.puts "WARNING: Could not load amber-lsp rule pack at #{pack_path}: #{ex.message}" + nil + end + + private def rule_pack_is_valid?(rule_pack : DescribeLibraryRulePack) : Bool + return false unless rule_pack.is_valid? + + rule_pack.modes_by_name.each_value do |mode| + mode.list_of_evidence_patterns.each { |pattern| Regex.new(pattern) } + end + + rule_pack.list_of_rules.all? do |rule| + next false unless {"error", "warning", "info", "hint"}.includes?(rule.severity_name.downcase) + + case rule.check.check_kind + when "line_regex" + Regex.new(rule.check.regex_pattern) + true + when "file_requires" + !rule.check.required_regex_pattern.empty? && + (rule.check.trigger_regex_pattern.empty? || begin + Regex.new(rule.check.trigger_regex_pattern) + true + end) && begin + Regex.new(rule.check.required_regex_pattern) + true + end + when "call_outside_block" + !rule.check.list_of_source_globs.empty? && + !rule.check.list_of_query_method_names.empty? && + (!rule.check.required_block_call_name.empty? || !rule.check.escape_block_call_name.empty?) + when "project_conflict" + case rule.check.project_condition + when "mixed_modes" + rule.list_of_mode_names.size > 1 + when "evidence_without_declaration" + rule.list_of_mode_names.size == 1 + else + false + end + else + false + end + end + rescue ArgumentError + false + end + end +end diff --git a/src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts.cr b/src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts.cr new file mode 100644 index 0000000..1d17509 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts.cr @@ -0,0 +1,90 @@ +module AmberLSP::LibraryRulePacks + class PrintDeclaredRulePackContexts + def initialize(@list_of_arguments : Array(String)) + end + + def perform : Int32 + project_context = AmberLSP::ProjectContext.detect(project_root_from_arguments) + list_of_rule_packs = LoadRulePacksForProject.new(project_context).load_rule_packs + list_of_output_lines = [] of String + + list_of_rule_packs.each do |rule_pack| + next if project_context.shard_name == rule_pack.library_shard_name + + project_state = DetermineProjectRulePackState.new(project_context, rule_pack, "", "") + next unless project_state.has_any_applicable_mode? + + append_declared_mode_contexts(list_of_output_lines, rule_pack, project_state) + append_undeclared_feature_warning(list_of_output_lines, rule_pack, project_state) + end + + STDOUT.puts(list_of_output_lines.join('\n')) unless list_of_output_lines.empty? + 0 + rescue ex + STDERR.puts "amber-lsp context failed: #{ex.message}" + 1 + end + + private def project_root_from_arguments : String + project_root = Dir.current + arguments = @list_of_arguments.dup + + until arguments.empty? + argument = arguments.shift + case argument + when "--root" + root_argument = arguments.shift? + raise ArgumentError.new("--root requires a directory") unless root_argument + project_root = File.expand_path(root_argument) + else + raise ArgumentError.new("unexpected argument #{argument.inspect}") + end + end + + find_project_root(project_root) + end + + private def find_project_root(start_path : String) : String + current_path = File.expand_path(start_path) + current_path = File.dirname(current_path) unless File.directory?(current_path) + + loop do + return current_path if File.file?(File.join(current_path, "shard.yml")) + + parent_path = File.dirname(current_path) + return current_path if parent_path == current_path + current_path = parent_path + end + end + + private def append_declared_mode_contexts( + list_of_output_lines : Array(String), + rule_pack : DescribeLibraryRulePack, + project_state : DetermineProjectRulePackState, + ) : Nil + rule_pack.modes_by_name.each do |mode_name, mode| + next unless project_state.is_mode_declared?(mode_name) + next if mode.guidance_text.strip.empty? + + list_of_output_lines << "#{rule_pack.pack_id} (#{mode_name})" + list_of_output_lines.concat(mode.guidance_text.lines.map(&.rstrip)) + end + end + + private def append_undeclared_feature_warning( + list_of_output_lines : Array(String), + rule_pack : DescribeLibraryRulePack, + project_state : DetermineProjectRulePackState, + ) : Nil + list_of_undeclared_mode_names = rule_pack.modes_by_name.keys.select do |mode_name| + project_state.has_evidence_for_mode?(mode_name) && !project_state.is_mode_declared?(mode_name) + end + return if list_of_undeclared_mode_names.empty? + + list_of_key_paths = list_of_undeclared_mode_names.compact_map do |mode_name| + rule_pack.modes_by_name[mode_name]?.try(&.declaration.key_path) + end + list_of_output_lines << "warning: #{rule_pack.pack_id} feature is used but shard.yml does not declare #{list_of_key_paths.join(", ")}." + end + end +end diff --git a/src/amber_lsp/library_rule_packs/visit_crystal_calls_outside_required_blocks.cr b/src/amber_lsp/library_rule_packs/visit_crystal_calls_outside_required_blocks.cr new file mode 100644 index 0000000..d5d7d5b --- /dev/null +++ b/src/amber_lsp/library_rule_packs/visit_crystal_calls_outside_required_blocks.cr @@ -0,0 +1,159 @@ +require "set" +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks + class VisitCrystalCallsOutsideRequiredBlocks < Crystal::Visitor + getter list_of_calls_outside_required_blocks : Array(Crystal::Call) + getter list_of_multitenant_model_names : Set(String) + + @list_of_class_names = [] of String + @required_block_depth = 0 + @list_of_unscoped_model_names = [] of String + @list_of_block_scope_changes = [] of {Bool, String?} + @list_of_method_scope_snapshots = [] of {Int32, Array(String)} + + def initialize( + @list_of_scoped_model_names : Set(String), + @list_of_query_method_names : Array(String), + @required_block_call_name : String, + @escape_block_call_name : String, + @model_macro_name : String, + ) + @list_of_calls_outside_required_blocks = [] of Crystal::Call + @list_of_multitenant_model_names = Set(String).new + end + + def self.find_multitenant_model_names(content : String, model_macro_name : String) : Set(String) + visitor = new( + Set(String).new, + [] of String, + "", + "", + model_macro_name, + ) + visitor.accept(Crystal::Parser.new(content).parse) + visitor.list_of_multitenant_model_names + rescue Crystal::SyntaxException + Set(String).new + end + + def visit(node : Crystal::ASTNode) : Bool + true + end + + def visit(node : Crystal::ClassDef) : Bool + @list_of_class_names << node.name.names.last + true + end + + def end_visit(node : Crystal::ClassDef) : Nil + @list_of_class_names.pop + end + + def visit(node : Crystal::Def) : Bool + @list_of_method_scope_snapshots << {@required_block_depth, @list_of_unscoped_model_names.dup} + @required_block_depth = 0 + @list_of_unscoped_model_names.clear + true + end + + def end_visit(node : Crystal::Def) : Nil + previous_scope = @list_of_method_scope_snapshots.pop? + return unless previous_scope + + @required_block_depth = previous_scope[0] + @list_of_unscoped_model_names = previous_scope[1] + end + + def visit(node : Crystal::Call) : Bool + if node.name == @model_macro_name && !@list_of_class_names.empty? + @list_of_multitenant_model_names << @list_of_class_names.last + end + + if query_call_is_outside_required_blocks?(node) + @list_of_calls_outside_required_blocks << node + end + + true + end + + def visit(node : Crystal::Block) : Bool + call = node.call + enters_required_block = call.try { |block_call| is_required_block_call?(block_call) } || false + if enters_required_block + @required_block_depth += 1 + end + + escape_model_name = call.try { |block_call| escaped_model_name(block_call) } + if escape_model_name + @list_of_unscoped_model_names << escape_model_name + end + + @list_of_block_scope_changes << {enters_required_block, escape_model_name} + true + end + + def end_visit(node : Crystal::Block) : Nil + scope_change = @list_of_block_scope_changes.pop? + return unless scope_change + + @required_block_depth -= 1 if scope_change[0] + if scope_change[1] + @list_of_unscoped_model_names.pop + end + end + + private def query_call_is_outside_required_blocks?(call : Crystal::Call) : Bool + return false unless @list_of_query_method_names.includes?(call.name) + + model_name = model_name_for_receiver(call.obj) + return false unless model_name && @list_of_scoped_model_names.includes?(model_name) + return false if @required_block_depth > 0 + return false if @list_of_unscoped_model_names.includes?(model_name) + + true + end + + private def is_required_block_call?(call : Crystal::Call) : Bool + return false if @required_block_call_name.empty? + + if @required_block_call_name.includes?('.') + full_call_name(call) == @required_block_call_name + else + call.name == @required_block_call_name + end + end + + private def escaped_model_name(call : Crystal::Call) : String? + return nil if @escape_block_call_name.empty? + return nil unless call.name == @escape_block_call_name + + model_name = model_name_for_receiver(call.obj) + return nil unless model_name && @list_of_scoped_model_names.includes?(model_name) + + model_name + end + + private def model_name_for_receiver(node : Crystal::ASTNode?) : String? + case node + when Crystal::Path + node.names.last? + when Crystal::Call + model_name_for_receiver(node.obj) + else + nil + end + end + + private def full_call_name(call : Crystal::Call) : String + case receiver = call.obj + when Crystal::Path + "#{receiver.names.join("::")}.#{call.name}" + when Crystal::Call + "#{full_call_name(receiver)}.#{call.name}" + else + call.name + end + end + end +end diff --git a/src/amber_lsp/project_context.cr b/src/amber_lsp/project_context.cr index 10b6d19..3064f9e 100644 --- a/src/amber_lsp/project_context.cr +++ b/src/amber_lsp/project_context.cr @@ -4,8 +4,16 @@ module AmberLSP class ProjectContext getter root_path : String getter? amber_project : Bool + getter shard_name : String? - def initialize(@root_path : String, @amber_project : Bool = false) + @shard_configuration : YAML::Any? + + def initialize( + @root_path : String, + @amber_project : Bool = false, + @shard_name : String? = nil, + @shard_configuration : YAML::Any? = nil, + ) end def self.detect(root_path : String) : ProjectContext @@ -15,20 +23,41 @@ module AmberLSP return ProjectContext.new(root_path, amber_project: false) end - content = File.read(shard_path) - is_amber = has_amber_dependency?(content) + shard_configuration = YAML.parse(File.read(shard_path)) + is_amber = has_amber_dependency?(shard_configuration) + shard_name = shard_configuration["name"]?.try(&.as_s?) + + ProjectContext.new( + root_path, + amber_project: is_amber, + shard_name: shard_name, + shard_configuration: shard_configuration, + ) + rescue YAML::ParseException + ProjectContext.new(root_path, amber_project: false) + end + + def has_shard_declaration?(key_path : String, expected_value : String) : Bool + shard_configuration = @shard_configuration + return false unless shard_configuration - ProjectContext.new(root_path, amber_project: is_amber) + current_value = shard_configuration + key_path.split('.').each do |key| + next_value = current_value[key]? + return false unless next_value + current_value = next_value + end + + current_value.as_s? == expected_value + rescue TypeCastError + false end - private def self.has_amber_dependency?(shard_content : String) : Bool - yaml = YAML.parse(shard_content) - dependencies = yaml["dependencies"]? + private def self.has_amber_dependency?(shard_configuration : YAML::Any) : Bool + dependencies = shard_configuration["dependencies"]? return false unless dependencies dependencies["amber"]? != nil - rescue YAML::ParseException - false end end end diff --git a/src/amber_lsp/rules/base_rule.cr b/src/amber_lsp/rules/base_rule.cr index 4e98963..6b2c1d3 100644 --- a/src/amber_lsp/rules/base_rule.cr +++ b/src/amber_lsp/rules/base_rule.cr @@ -6,6 +6,10 @@ module AmberLSP::Rules abstract def applies_to : Array(String) abstract def check(file_path : String, content : String) : Array(Diagnostic) + def requires_amber_project? : Bool + true + end + # Finds the line and character range for the first occurrence of a pattern. # Returns nil if the pattern is not found. def find_line_range(content : String, pattern : Regex) : TextRange? diff --git a/src/amber_lsp/rules/controllers/action_return_rule.cr b/src/amber_lsp/rules/controllers/action_return_rule.cr index b09a090..00d638c 100644 --- a/src/amber_lsp/rules/controllers/action_return_rule.cr +++ b/src/amber_lsp/rules/controllers/action_return_rule.cr @@ -3,6 +3,7 @@ module AmberLSP::Rules::Controllers RESPONSE_METHODS = ["render", "redirect_to", "redirect_back", "respond_with", "halt!"] SKIPPED_METHODS = ["initialize", "before_action", "after_action", "before_filter", "after_filter"] VISIBILITY_CHANGE = /^\s*(private|protected)\s*$/ + CLASS_METHOD = /^\s{2,4}def\s+self\./ def id : String "amber/action-return-type" @@ -22,6 +23,7 @@ module AmberLSP::Rules::Controllers def check(file_path : String, content : String) : Array(Diagnostic) return [] of Diagnostic unless file_path.includes?("controllers/") + return [] of Diagnostic if action_support_file?(file_path) diagnostics = [] of Diagnostic lines = content.lines @@ -45,6 +47,8 @@ module AmberLSP::Rules::Controllers # Detect method start at standard 2-space indent (methods inside a class) method_match = /^(\s{2,4})def\s+(\w+)/.match(line) if method_match && !in_public_method + next if CLASS_METHOD.matches?(line) + indent = method_match[1].size name = method_match[2] @@ -65,7 +69,7 @@ module AmberLSP::Rules::Controllers if in_public_method # Check for response method calls - if RESPONSE_METHODS.any? { |m| line.includes?(m) } + if response_written_or_returned?(line) has_response_call = true end @@ -93,6 +97,19 @@ module AmberLSP::Rules::Controllers diagnostics end + + private def action_support_file?(file_path : String) : Bool + File.basename(file_path) == "application_controller.cr" || + file_path.includes?("controllers/concerns/") + end + + private def response_written_or_returned?(line : String) : Bool + RESPONSE_METHODS.any? { |method_name| line.includes?(method_name) } || + line.includes?("respond_") || + line.matches?(/\bcontext\.response\.(print|write)\b/) || + line.matches?(/\b\w+\(context\.response\b/) || + line.includes?("String.build") + end end end diff --git a/src/amber_lsp/rules/custom_rule.cr b/src/amber_lsp/rules/custom_rule.cr index aafb447..04b028b 100644 --- a/src/amber_lsp/rules/custom_rule.cr +++ b/src/amber_lsp/rules/custom_rule.cr @@ -20,6 +20,10 @@ module AmberLSP::Rules ) end + def requires_amber_project? : Bool + false + end + def check(file_path : String, content : String) : Array(Diagnostic) return [] of Diagnostic unless applies_to.any? { |pattern| RuleRegistry.file_matches_pattern?(file_path, pattern) diff --git a/src/amber_lsp/rules/rule_registry.cr b/src/amber_lsp/rules/rule_registry.cr index b43490a..d0e1c1e 100644 --- a/src/amber_lsp/rules/rule_registry.cr +++ b/src/amber_lsp/rules/rule_registry.cr @@ -23,6 +23,11 @@ module AmberLSP::Rules def self.file_matches_pattern?(file_path : String, pattern : String) : Bool if pattern == "*" true + elsif File.match?(pattern, file_path) + true + elsif pattern.starts_with?("*") && !pattern.includes?("/") + # Preserve the existing suffix behavior for rules such as *_controller.cr. + file_path.ends_with?(pattern.lchop("*")) elsif pattern.ends_with?("**") # Recursive glob: "src/**" matches anything under "src/" prefix = pattern.rchop("**") From 5a43008bdf1d2326025222df5fa419bc7e1a16a5 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Wed, 23 Sep 2026 20:47:39 -0400 Subject: [PATCH 02/69] feat(amber-lsp): detect Grant tenancy from AST rule packs --- docs/rule-packs.md | 111 ++- spec/amber_lsp/rule_packs_spec.cr | 707 ++++++++++-------- spec/amber_lsp/spec_helper.cr | 10 +- .../mixed_app/config/tenant_scope.cr | 3 + .../mixed_app/src/models/invoice.cr | 8 + .../no_tenancy_app/src/models/invoice.cr | 6 + .../lib/grant/.amber-lsp/packs/tenancy.yml | 142 ++++ .../row_app/src/models/account.cr | 6 + .../row_app/src/models/custom_document.cr | 5 + .../row_app/src/models/invoice.cr | 8 + .../row_app/src/models/invoice_export.cr | 4 + .../row_app/src/models/ledger_entry.cr | 5 + .../schema_app/config/tenant_scope.cr | 3 + .../schema_app/src/models/country.cr | 7 + .../schema_app/src/models/invoice.cr | 5 + src/amber_lsp.cr | 12 +- .../analyze_project_files_with_rule_packs.cr | 135 +++- .../describe_library_rule_pack.cr | 19 +- .../determine_project_rule_pack_state.cr | 204 +++-- ...lect_project_grant_tenancy_declarations.cr | 157 ++++ .../grant_tenant_model_declaration.cr | 50 ++ .../grant_tenancy/source_node.cr | 79 ++ .../visit_chainable_unscoped_model_calls.cr | 73 ++ ...nt_schema_queries_outside_tenant_blocks.cr | 77 ++ .../visit_grant_tenant_clear_calls.cr | 22 + .../visit_raw_connection_sql_call_sites.cr | 123 +++ ..._spawn_calls_inside_grant_tenant_blocks.cr | 62 ++ .../load_rule_packs_for_project.cr | 31 +- ...r => print_detected_rule_pack_contexts.cr} | 25 +- src/amber_lsp/project_context.cr | 20 - 30 files changed, 1615 insertions(+), 504 deletions(-) create mode 100644 spec/fixtures/rule_pack_apps/mixed_app/config/tenant_scope.cr create mode 100644 spec/fixtures/rule_pack_apps/mixed_app/src/models/invoice.cr create mode 100644 spec/fixtures/rule_pack_apps/no_tenancy_app/src/models/invoice.cr create mode 100644 spec/fixtures/rule_pack_apps/row_app/lib/grant/.amber-lsp/packs/tenancy.yml create mode 100644 spec/fixtures/rule_pack_apps/row_app/src/models/account.cr create mode 100644 spec/fixtures/rule_pack_apps/row_app/src/models/custom_document.cr create mode 100644 spec/fixtures/rule_pack_apps/row_app/src/models/invoice.cr create mode 100644 spec/fixtures/rule_pack_apps/row_app/src/models/invoice_export.cr create mode 100644 spec/fixtures/rule_pack_apps/row_app/src/models/ledger_entry.cr create mode 100644 spec/fixtures/rule_pack_apps/schema_app/config/tenant_scope.cr create mode 100644 spec/fixtures/rule_pack_apps/schema_app/src/models/country.cr create mode 100644 spec/fixtures/rule_pack_apps/schema_app/src/models/invoice.cr create mode 100644 src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations.cr create mode 100644 src/amber_lsp/library_rule_packs/grant_tenancy/grant_tenant_model_declaration.cr create mode 100644 src/amber_lsp/library_rule_packs/grant_tenancy/source_node.cr create mode 100644 src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls.cr create mode 100644 src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_schema_queries_outside_tenant_blocks.cr create mode 100644 src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_tenant_clear_calls.cr create mode 100644 src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites.cr create mode 100644 src/amber_lsp/library_rule_packs/grant_tenancy/visit_spawn_calls_inside_grant_tenant_blocks.cr rename src/amber_lsp/library_rule_packs/{print_declared_rule_pack_contexts.cr => print_detected_rule_pack_contexts.cr} (68%) diff --git a/docs/rule-packs.md b/docs/rule-packs.md index 2a2b662..834b1bb 100644 --- a/docs/rule-packs.md +++ b/docs/rule-packs.md @@ -1,92 +1,133 @@ # Library rule packs -Library rule packs let a Crystal library ship opt-in feature guidance and edit-time checks with the same `.claude/` files that Shards distributes. `amber-lsp` reads packs from installed dependencies at `lib/*/.claude/rules/*.yml` and from the project at `.claude/rules/*.yml`; no copy step is required. A project pack with the same `pack` id overrides a dependency pack. A pack whose `library` equals the project's own shard name does not apply, because that pack describes the library's consumer contract. +Library rule packs let a Crystal library ship feature guidance and edit-time +checks for `amber-lsp`, a harness-neutral tool. The pack uses amber-lsp's YAML +format and lives in the library at `.amber-lsp/packs/.yml`. The engine +discovers installed dependency packs at `lib/*/.amber-lsp/packs/*.yml` and +project packs at `.amber-lsp/packs/*.yml`. A project pack with the same `pack` +id overrides a dependency pack. A pack whose `library` equals the project's +own shard name does not apply, because that pack describes the library's +consumer contract. ## Pack document -Each YAML file has a pack identity, declared modes, optional evidence patterns, mode-specific context, and rules: +Each YAML file has a pack identity, source-detected modes with context, and +rules: ```yaml pack: grant/tenancy library: grant -version: 1.0.0 +version: 2.0.0 modes: row: - declared_by: - key_path: grant.tenancy - expected_value: row - evidence: - - '^\s*multitenant\b' - tenant_column: tenant_id context: | - Scope queries with the library's tenant context. + Row tenancy was detected from the app's multitenant model macros. schema: - declared_by: - key_path: grant.tenancy - expected_value: schema - evidence: - - '^\s*SchemaTenant\.with\b' context: | - Schema-specific rules are not included yet. + Schema tenancy was detected from the app's schema-tenant declarations. rules: - id: grant/example modes: [row] severity: warning applies_to: ["src/**"] - exclude_from: ["src/controllers/**"] + exclude_from: ["spec/**"] message: "Explain the finding and the repair." check: - kind: line_regex - pattern: '\bunsafe_call\b' + kind: crystal_ast + operation: chained_unscoped_on_tenant_model ``` -`declared_by.key_path` is a dotted path into the application's `shard.yml`; `expected_value` is compared as a string. A pack rule runs only when one of its modes is declared. Evidence patterns scan the application's `.cr` files outside `lib/`, `.git/`, `tmp/`, and `vendor/`; they support mixed-mode detection and the undeclared-feature warning. Evidence alone does not activate mode-specific rules. Context blocks are printed only for declared modes and should stay to about 15 lines or fewer. - -Each rule requires an id, one or more mode names, a severity, file globs, a message, and one check. Severity values are `error`, `warning`, `info`, and `hint`. `exclude_from` is optional. `applies_to` and `exclude_from` use Crystal path globs, including `**` for recursive directories. Rules can be enabled, disabled, or have their severity overridden through the existing `.amber-lsp.yml` `rules` mapping. +For Grant, `amber-lsp` parses Crystal source under the app's `src/` and +`config/` directories to detect tenancy. It does not scan installed `lib/` code +or `spec/` files for mode declarations. Row mode comes from each class body's +`multitenant :column` macro, and the engine captures that tenant column per +model. Schema mode comes from a `Grant::SchemaTenant.with(...)` call or a +`schema_tenant_excluded` macro in a class body. Comments and string contents do +not count as declarations, and an unparsable source file is skipped. + +Rules run only when one of their listed modes is detected. Context blocks are +printed by `amber-lsp context [--root DIR]` for detected modes. There is no +shard.yml feature declaration or evidence-without-declaration warning. The +engine reads shard.yml only for package identity and installed dependency +names. + +Each rule requires an id, one or more mode names, a severity, file globs, a +message, and one check. Severity values are `error`, `warning`, `info`, and +`hint`. `exclude_from` is optional. `applies_to` and `exclude_from` use Crystal +path globs, including `**` for recursive directories. Rules can be enabled, +disabled, or have their severity overridden through the existing +`.amber-lsp.yml` `rules` mapping. ## Check kinds ### `line_regex` -Matches a regular expression on each line and uses the existing custom-rule diagnostic matcher. `negate: true` reports once at the start of a file when the pattern does not occur. +Matches a regular expression on each line using the existing custom-rule +diagnostic matcher. `negate: true` reports once at the start of a file when the +pattern does not occur. ```yaml check: kind: line_regex - pattern: '^\s*[^#]*\.unscoped\b' + pattern: '^\\s*[^#]*\\.unsafe_call\\b' ``` ### `file_requires` -Reports on each line matching `trigger_pattern` when the file has no line matching `required_pattern`. If `trigger_pattern` is omitted, `amber-lsp` uses the first mode's `tenant_column` as a Crystal `column ` declaration. +Reports on each line matching `trigger_pattern` when the file has no line +matching `required_pattern`. Both are explicit regular expressions. ```yaml check: kind: file_requires - required_pattern: '^\s*multitenant\b' + trigger_pattern: '^\\s*column\\s+account_id\\b' + required_pattern: '^\\s*multitenant\\b' ``` ### `call_outside_block` -Uses `Crystal::Parser` and a `Crystal::Visitor` to inspect method calls. `source_globs` select the files where `tenant_macro` declarations identify scoped model names. `methods` lists calls on those model constants that can query the database. A dotted `required_call`, such as `Grant::Tenant.with`, permits calls lexically inside its block and nested blocks. `escape_call` permits calls only inside a block on that same model receiver, such as `Todo.unscoped { Todo.where(...) }`. A method definition starts a new lexical scope, so a query written in a method body does not become tenant-scoped just because the method definition appears inside a tenant block. Receiver constants are followed through chained calls, but model instances held in variables are not resolved. The current matcher uses the last constant name, so same-named models in different namespaces can be ambiguous. +Uses `Crystal::Parser` and a `Crystal::Visitor` to inspect method calls. +`source_globs` select the files where `tenant_macro` declarations identify +scoped model names. `methods` lists calls on those model constants that can +query the database. A dotted `required_call`, such as `Grant::Tenant.with`, +permits calls lexically inside its block and nested blocks. `escape_call` +permits calls only inside a block on that same model receiver, such as +`Todo.unscoped { Todo.where(...) }`. A method definition starts a new lexical +scope. Receiver constants are followed through chained calls, but model +instances held in variables are not resolved. ```yaml check: kind: call_outside_block source_globs: ["src/models/**"] tenant_macro: multitenant - methods: [where, find, all, first, count] + methods: [where, find, all] required_call: Grant::Tenant.with escape_call: unscoped ``` -`required_call` may be omitted when `escape_call` alone defines the allowed block. The rule intentionally does not infer receiver types or follow aliases stored in local variables. +### `crystal_ast` + +Runs one of the named AST operations used by Grant's tenancy pack. The parser +visitors inspect Crystal call, class, block, and literal nodes rather than +matching source lines. Supported operations are `chained_unscoped_in_request_code`, +`chained_unscoped_bulk_write`, `chained_unscoped_on_tenant_model`, +`unscoped_block_in_request_code`, `spawn_inside_tenant_block`, +`tenant_column_without_multitenant`, `raw_connection_sql_on_tenant_table`, +`tenant_clear_in_app_code`, and `schema_query_outside_tenant`. + +```yaml +check: + kind: crystal_ast + operation: tenant_column_without_multitenant +``` ### `project_conflict` -Runs over project state rather than one source line. The default `condition: mixed_modes` reports when every listed mode is declared or has evidence. `condition: evidence_without_declaration` reports when a listed mode has feature evidence but its declaration is absent; use one mode for this warning. +Runs over project state. The supported `condition: mixed_modes` reports when +both listed modes are detected in the app's Crystal source. ```yaml check: @@ -96,6 +137,12 @@ check: ## Project activation and context -Amber's built-in convention rules remain gated on an `amber` dependency. Library packs run in any project when the pack applies, including a project with only a Grant dependency. Pack diagnostics use the ordinary LSP diagnostic channel and `.amber-lsp.yml` severity overrides. +Amber's built-in convention rules remain gated on an `amber` dependency. +Library packs run in any project when a pack mode is detected, including a +non-Amber app with Grant installed. Pack diagnostics use the ordinary LSP +diagnostic channel and `.amber-lsp.yml` severity overrides. -Run `amber-lsp context [--root DIR]` to print the context blocks for declared pack modes and one warning line per pack whose feature is used without a declaration. It prints nothing when no pack is declared or evidenced. Exit status is 0 for successful context inspection; invalid command arguments or an unreadable project return nonzero. +Run `amber-lsp context [--root DIR]` to print the context blocks for detected +pack modes. It prints nothing when no mode is detected. Exit status is 0 for +successful context inspection; invalid command arguments or an unreadable +project return nonzero. diff --git a/spec/amber_lsp/rule_packs_spec.cr b/spec/amber_lsp/rule_packs_spec.cr index 2d9f4c1..51ebe49 100644 --- a/spec/amber_lsp/rule_packs_spec.cr +++ b/spec/amber_lsp/rule_packs_spec.cr @@ -1,423 +1,492 @@ require "./spec_helper" -require "../../src/amber_lsp/rules/controllers/action_return_rule" - -GRANT_RULE_PACK_FIXTURE = <<-YAML - pack: grant/tenancy - library: grant - version: 1.0.0 - modes: - row: - declared_by: - key_path: grant.tenancy - expected_value: row - evidence: - - '^\\s*multitenant\\b' - tenant_column: tenant_id - context: | - Scope queries with Grant::Tenant.with. - schema: - declared_by: - key_path: grant.tenancy - expected_value: schema - evidence: - - '^\\s*Grant::SchemaTenant\\.with\\b' - - '^\\s*schema_tenant_excluded\\b' - context: | - Schema-specific rules are not included yet. - rules: - - id: grant/tenant-column-without-multitenant - modes: [row] - severity: error - applies_to: ["src/models/**"] - message: Tenant columns require multitenant. - check: - kind: file_requires - required_pattern: '^\\s*multitenant\\b' - - id: grant/tenancy-modes-mixed - modes: [row, schema] - severity: error - applies_to: ["**/*.cr"] - message: Choose one tenancy mode. - check: - kind: project_conflict - condition: mixed_modes - - id: grant/row-query-outside-tenant - modes: [row] - severity: warning - applies_to: ["**/*.cr"] - exclude_from: ["src/controllers/**"] - message: Query must be tenant-scoped. - check: - kind: call_outside_block - source_globs: ["src/models/**"] - tenant_macro: multitenant - methods: [all, where, find!] - required_call: Grant::Tenant.with - escape_call: unscoped - - id: grant/unscoped-in-request-code - modes: [row] - severity: warning - applies_to: ["src/controllers/**"] - message: Do not use unscoped in request code. - check: - kind: line_regex - pattern: '^\\s*[^#]*\\.unscoped\\b' - - id: grant/raw-sql-on-scoped-model - modes: [row] - severity: warning - applies_to: ["**/*.cr"] - message: Raw SQL must use an unscoped block. - check: - kind: call_outside_block - source_globs: ["src/models/**"] - tenant_macro: multitenant - methods: [exec, query, scalar] - escape_call: unscoped - - id: grant/tenancy-undeclared - modes: [row] - severity: warning - applies_to: ["**/*.cr"] - message: Declare Grant tenancy in shard.yml. - check: - kind: project_conflict - condition: evidence_without_declaration - YAML - -def write_rule_pack_project( - root : String, - shard_content : String = "name: tenant_app\nversion: 0.1.0\ngrant:\n tenancy: row\n", - pack_content : String = GRANT_RULE_PACK_FIXTURE, -) : Nil - Dir.mkdir_p(File.join(root, ".claude", "rules")) - File.write(File.join(root, "shard.yml"), shard_content) - File.write(File.join(root, ".claude", "rules", "tenancy.yml"), pack_content) + +GRANT_TENANCY_PACK_FIXTURE_PATH = File.join( + Dir.current, + "spec", + "fixtures", + "rule_pack_apps", + "row_app", + "lib", + "grant", + ".amber-lsp", + "packs", + "tenancy.yml", +) + +GRANT_TENANCY_REQUEST_PATHS = [ + "src/controllers/invoices_controller.cr", + "src/channels/invoice_channel.cr", + "src/sockets/invoice_socket.cr", + "src/pipes/tenant_pipe.cr", +] + +def install_tenancy_fixture_app(root : String, fixture_name : String) : Nil + fixture_root = File.join(Dir.current, "spec", "fixtures", "rule_pack_apps", fixture_name) + + ["src", "config", "lib"].each do |directory_name| + FileUtils.rm_rf(File.join(root, directory_name)) + end + + File.write(File.join(root, "shard.yml"), "name: #{fixture_name}\nversion: 0.1.0\n") + + ["src", "config"].each do |source_directory_name| + source_directory = File.join(fixture_root, source_directory_name) + next unless Dir.exists?(source_directory) + + FileUtils.cp_r(source_directory, root) + end + + pack_path = File.join(root, "lib", "grant", ".amber-lsp", "packs", "tenancy.yml") + Dir.mkdir_p(File.dirname(pack_path)) + File.write(pack_path, File.read(GRANT_TENANCY_PACK_FIXTURE_PATH)) end -def analyze_pack_file(root : String, file_path : String, content : String) : Array(AmberLSP::Rules::Diagnostic) +def analyze_tenancy_fixture_source( + root : String, + relative_file_path : String, + content : String, +) : Array(AmberLSP::Rules::Diagnostic) + file_path = File.join(root, relative_file_path) Dir.mkdir_p(File.dirname(file_path)) File.write(file_path, content) + project_context = AmberLSP::ProjectContext.detect(root) analyzer = AmberLSP::Analyzer.new analyzer.configure(project_context) analyzer.analyze(file_path, content) end -def diagnostic_codes(diagnostics : Array(AmberLSP::Rules::Diagnostic)) : Array(String) +def has_tenancy_diagnostic_code?(diagnostics : Array(AmberLSP::Rules::Diagnostic), code : String) : Bool + diagnostics.any? { |diagnostic| diagnostic.code == code } +end + +def tenancy_diagnostic_codes(diagnostics : Array(AmberLSP::Rules::Diagnostic)) : Array(String) diagnostics.map(&.code) end -describe "AmberLSP library rule packs" do +describe "AmberLSP Grant tenancy rule pack v2" do before_each do AmberLSP::Rules::RuleRegistry.clear end - it "loads packs from installed dependencies and project rules" do + it "loads the library pack from the harness-neutral dependency path" do with_tempdir do |root| - write_rule_pack_project(root) - dependency_root = File.join(root, "dependency_source") - dependency_pack_path = File.join(dependency_root, ".claude", "rules", "tenancy.yml") - project_pack_path = File.join(root, ".claude", "rules", "project.yml") - Dir.mkdir_p(File.dirname(dependency_pack_path)) - File.write(dependency_pack_path, GRANT_RULE_PACK_FIXTURE) - Dir.mkdir_p(File.join(root, "lib")) - File.symlink(dependency_root, File.join(root, "lib", "grant")) - File.write( - project_pack_path, - GRANT_RULE_PACK_FIXTURE.gsub("grant/tenancy", "project/tenancy").gsub("library: grant", "library: project"), - ) + install_tenancy_fixture_app(root, "row_app") project_context = AmberLSP::ProjectContext.detect(root) - packs = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs + list_of_rule_packs = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs - packs.map(&.pack_id).should eq(["grant/tenancy", "project/tenancy"]) + list_of_rule_packs.map(&.pack_id).should eq(["grant/tenancy"]) + File.exists?(File.join(root, "lib", "grant", ".amber-lsp", "packs", "tenancy.yml")).should be_true + File.exists?(File.join(root, "lib", "grant", ".claude", "rules", "tenancy.yml")).should be_false end end - it "keeps a library's own pack inactive while allowing clean LSP checks" do + it "gives agents the Grant runtime context for detected modes" do with_tempdir do |root| - shard_content = "name: grant\nversion: 0.1.0\n" - write_rule_pack_project(root, shard_content) - + install_tenancy_fixture_app(root, "row_app") project_context = AmberLSP::ProjectContext.detect(root) - packs = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs - packs.map(&.pack_id).should eq(["grant/tenancy"]) - - file_path = File.join(root, "src", "grant", "scale", "tenant.cr") - content = "multitenant :tenant_id\n" - analyzer = AmberLSP::Analyzer.new - analyzer.configure(project_context) - analyzer.has_applicable_library_rule_pack?(file_path, content).should be_true - analyzer.analyze(file_path, content).should be_empty + rule_pack = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs.first + + rule_pack.should_not be_nil + if loaded_rule_pack = rule_pack + row_context = loaded_rule_pack.modes_by_name["row"].guidance_text + row_context.should contain("detected from the app's `multitenant` model macros") + row_context.should contain("ScopedRawSqlError") + row_context.should contain("fiber-local") + schema_context = loaded_rule_pack.modes_by_name["schema"].guidance_text + schema_context.should contain("default search path without an error") + schema_context.should contain("PostgreSQL") + end end end - it "runs a declared dependency pack in a non-Amber project" do + it "loads exactly ten rules with only the two intended errors" do with_tempdir do |root| - write_rule_pack_project(root) - Dir.mkdir_p(File.join(root, "src", "jobs")) - file_path = File.join(root, "src", "jobs", "fixture.cr") - diagnostics = analyze_pack_file(root, file_path, "puts \"unscoped\"\n") - - diagnostics.map(&.code).should eq([] of String) + install_tenancy_fixture_app(root, "row_app") + project_context = AmberLSP::ProjectContext.detect(root) + loaded_rule_pack = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs.first + + loaded_rule_pack.should_not be_nil + if rule_pack = loaded_rule_pack + rule_pack.list_of_rules.size.should eq(10) + rule_pack.list_of_rules.count { |rule| rule.severity_name == "error" }.should eq(2) + rule_pack.list_of_rules.count { |rule| rule.severity_name == "warning" }.should eq(8) + end end end - it "runs line_regex checks and ignores full-line comments" do + it "runs a pack in a non-Amber app and detects the account_id macro argument" do with_tempdir do |root| - write_rule_pack_project(root) - file_path = File.join(root, "src", "controllers", "todos_controller.cr") - content = "# Todo.unscoped is only documentation\nTodo.unscoped.all\n" + install_tenancy_fixture_app(root, "row_app") + project_context = AmberLSP::ProjectContext.detect(root) + project_context.amber_project?.should be_false - diagnostics = analyze_pack_file(root, file_path, content) + diagnostics = analyze_tenancy_fixture_source( + root, + "src/controllers/invoices_controller.cr", + "Invoice.unscoped.all\n", + ) - diagnostics.map(&.code).should eq(["grant/unscoped-in-request-code"]) - diagnostics.first.range.start.line.should eq(1) - diagnostics.first.severity.should eq(AmberLSP::Rules::Severity::Warning) + has_tenancy_diagnostic_code?(diagnostics, "grant/chained-unscoped-in-request-code").should be_true + diagnostics.any? do |diagnostic| + diagnostic.code == "grant/chained-unscoped-in-request-code" && + diagnostic.severity == AmberLSP::Rules::Severity::Error + end.should be_true end end - it "reports a tenant column without multitenant and accepts the declared macro" do + it "reports chainable unscoped calls in each request path and allows block-form unscoped" do with_tempdir do |root| - write_rule_pack_project(root) - file_path = File.join(root, "src", "models", "account.cr") - Dir.mkdir_p(File.dirname(file_path)) - missing_macro = "class Account\n column tenant_id : Int64\nend\n" + install_tenancy_fixture_app(root, "row_app") + + GRANT_TENANCY_REQUEST_PATHS.each do |relative_path| + diagnostics = analyze_tenancy_fixture_source(root, relative_path, "Invoice.unscoped.all\n") + has_tenancy_diagnostic_code?(diagnostics, "grant/chained-unscoped-in-request-code").should be_true + + block_diagnostics = analyze_tenancy_fixture_source( + root, + relative_path, + "Invoice.unscoped { Invoice.all }\n", + ) + has_tenancy_diagnostic_code?(block_diagnostics, "grant/chained-unscoped-in-request-code").should be_false + end + end + end - diagnostics = analyze_pack_file(root, file_path, missing_macro) + it "reports unscoped bulk writes and leaves scoped writes clean" do + with_tempdir do |root| + install_tenancy_fixture_app(root, "row_app") - diagnostics.map(&.code).should contain("grant/tenant-column-without-multitenant") - diagnostics.find(&.code.==("grant/tenant-column-without-multitenant")).not_nil!.severity.should eq(AmberLSP::Rules::Severity::Error) + bulk_write_diagnostics = analyze_tenancy_fixture_source( + root, + "src/jobs/delete_invoices_job.cr", + "Invoice.unscoped.delete_all\n", + ) + has_tenancy_diagnostic_code?(bulk_write_diagnostics, "grant/chained-unscoped-bulk-write").should be_true + bulk_write_diagnostics.any? do |diagnostic| + diagnostic.code == "grant/chained-unscoped-bulk-write" && diagnostic.severity == AmberLSP::Rules::Severity::Error + end.should be_true + + scoped_write_diagnostics = analyze_tenancy_fixture_source( + root, + "src/jobs/delete_invoices_job.cr", + "Invoice.where(id: 1).delete_all\n", + ) + has_tenancy_diagnostic_code?(scoped_write_diagnostics, "grant/chained-unscoped-bulk-write").should be_false - valid_model = "class Account\n column tenant_id : Int64\n multitenant :tenant_id\nend\n" - valid_diagnostics = analyze_pack_file(root, file_path, valid_model) + spec_diagnostics = analyze_tenancy_fixture_source( + root, + "spec/delete_invoices_spec.cr", + "Invoice.unscoped.update_all({\"status\" => \"archived\"})\n", + ) + has_tenancy_diagnostic_code?(spec_diagnostics, "grant/chained-unscoped-bulk-write").should be_false - valid_diagnostics.map(&.code).should_not contain("grant/tenant-column-without-multitenant") + db_diagnostics = analyze_tenancy_fixture_source( + root, + "db/backfill.cr", + "Invoice.unscoped.delete_all\n", + ) + has_tenancy_diagnostic_code?(db_diagnostics, "grant/chained-unscoped-bulk-write").should be_false end end - it "uses the configured tenant column for file_requires" do + it "warns about chainable unscoped reads outside request code and excludes bulk writes" do with_tempdir do |root| - pack_content = GRANT_RULE_PACK_FIXTURE.gsub("tenant_column: tenant_id", "tenant_column: account_id") - write_rule_pack_project(root, pack_content: pack_content) - file_path = File.join(root, "src", "models", "account.cr") - Dir.mkdir_p(File.dirname(file_path)) + install_tenancy_fixture_app(root, "row_app") + + read_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/read_invoices_job.cr", "Invoice.unscoped.all\n") + has_tenancy_diagnostic_code?(read_diagnostics, "grant/chained-unscoped-on-tenant-model").should be_true + + request_diagnostics = analyze_tenancy_fixture_source( + root, + "src/controllers/invoices_controller.cr", + "Invoice.unscoped.all\n", + ) + has_tenancy_diagnostic_code?(request_diagnostics, "grant/chained-unscoped-on-tenant-model").should be_false - diagnostics = analyze_pack_file(root, file_path, "class Account\n column account_id : Int64\nend\n") + write_diagnostics = analyze_tenancy_fixture_source( + root, + "src/jobs/delete_invoices_job.cr", + "Invoice.unscoped.delete_all\n", + ) + has_tenancy_diagnostic_code?(write_diagnostics, "grant/chained-unscoped-on-tenant-model").should be_false - diagnostics.map(&.code).should contain("grant/tenant-column-without-multitenant") + scoped_read_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/read_invoices_job.cr", "Invoice.all\n") + has_tenancy_diagnostic_code?(scoped_read_diagnostics, "grant/chained-unscoped-on-tenant-model").should be_false end end - it "reports a multitenant query outside the required block" do + it "warns about block-form unscoped in request code and allows the default scope" do with_tempdir do |root| - write_rule_pack_project(root) - Dir.mkdir_p(File.join(root, "src", "models")) - File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") - file_path = File.join(root, "src", "jobs", "cleanup_job.cr") - Dir.mkdir_p(File.dirname(file_path)) + install_tenancy_fixture_app(root, "row_app") - diagnostics = analyze_pack_file(root, file_path, "Todo.where(active: true)\n") + unsafe_diagnostics = analyze_tenancy_fixture_source( + root, + "src/controllers/invoices_controller.cr", + "Invoice.unscoped { Invoice.all }\n", + ) + has_tenancy_diagnostic_code?(unsafe_diagnostics, "grant/unscoped-block-in-request-code").should be_true - diagnostics.map(&.code).should contain("grant/row-query-outside-tenant") + safe_diagnostics = analyze_tenancy_fixture_source( + root, + "src/controllers/invoices_controller.cr", + "Invoice.all\n", + ) + has_tenancy_diagnostic_code?(safe_diagnostics, "grant/unscoped-block-in-request-code").should be_false end end - it "accepts queries inside the required block and nested blocks" do + it "warns when spawn is inside either tenant block and allows spawn outside" do with_tempdir do |root| - write_rule_pack_project(root) - Dir.mkdir_p(File.join(root, "src", "models")) - File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") - file_path = File.join(root, "src", "jobs", "cleanup_job.cr") - Dir.mkdir_p(File.dirname(file_path)) - content = <<-CRYSTAL - Grant::Tenant.with(7) do - Todo.where(active: true) - run do - Todo.all - end - end - CRYSTAL + install_tenancy_fixture_app(root, "row_app") + row_diagnostics = analyze_tenancy_fixture_source( + root, + "src/jobs/invoice_job.cr", + "Grant::Tenant.with(7) { spawn { Invoice.all } }\n", + ) + has_tenancy_diagnostic_code?(row_diagnostics, "grant/spawn-inside-tenant-block").should be_true - diagnostics = analyze_pack_file(root, file_path, content) + outside_diagnostics = analyze_tenancy_fixture_source( + root, + "src/jobs/invoice_job.cr", + "spawn { Invoice.all }\n", + ) + has_tenancy_diagnostic_code?(outside_diagnostics, "grant/spawn-inside-tenant-block").should be_false - diagnostics.map(&.code).should_not contain("grant/row-query-outside-tenant") + install_tenancy_fixture_app(root, "schema_app") + schema_diagnostics = analyze_tenancy_fixture_source( + root, + "src/jobs/invoice_job.cr", + "Grant::SchemaTenant.with(\"acme\") { spawn { Invoice.all } }\n", + ) + has_tenancy_diagnostic_code?(schema_diagnostics, "grant/spawn-inside-tenant-block").should be_true end end - it "does not treat a method defined inside a tenant block as scoped" do + it "uses captured tenant columns and skips a model whose table the column references" do with_tempdir do |root| - write_rule_pack_project(root) - Dir.mkdir_p(File.join(root, "src", "models")) - File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") - file_path = File.join(root, "src", "jobs", "cleanup_job.cr") - Dir.mkdir_p(File.dirname(file_path)) - content = <<-CRYSTAL - Grant::Tenant.with(7) do - def load_todos - Todo.all - end - end - CRYSTAL - - diagnostics = analyze_pack_file(root, file_path, content) - - diagnostics.map(&.code).should contain("grant/row-query-outside-tenant") + install_tenancy_fixture_app(root, "row_app") + + missing_macro = File.read(File.join( + Dir.current, + "spec", + "fixtures", + "rule_pack_apps", + "row_app", + "src", + "models", + "invoice_export.cr", + )) + missing_diagnostics = analyze_tenancy_fixture_source(root, "src/models/invoice_export.cr", missing_macro) + has_tenancy_diagnostic_code?(missing_diagnostics, "grant/tenant-column-without-multitenant").should be_true + missing_diagnostics.any? do |diagnostic| + diagnostic.code == "grant/tenant-column-without-multitenant" && diagnostic.message.includes?("account_id") + end.should be_true + + declared_macro = missing_macro.sub("column account_id : Int64", "column account_id : Int64\n multitenant :account_id") + declared_diagnostics = analyze_tenancy_fixture_source(root, "src/models/invoice_export.cr", declared_macro) + has_tenancy_diagnostic_code?(declared_diagnostics, "grant/tenant-column-without-multitenant").should be_false + + referenced_table = File.read(File.join( + Dir.current, + "spec", + "fixtures", + "rule_pack_apps", + "row_app", + "src", + "models", + "account.cr", + )) + referenced_diagnostics = analyze_tenancy_fixture_source(root, "src/models/account.cr", referenced_table) + has_tenancy_diagnostic_code?(referenced_diagnostics, "grant/tenant-column-without-multitenant").should be_false + + unrelated_column = missing_macro.sub("account_id", "owner_id") + unrelated_diagnostics = analyze_tenancy_fixture_source(root, "src/models/invoice_export.cr", unrelated_column) + has_tenancy_diagnostic_code?(unrelated_diagnostics, "grant/tenant-column-without-multitenant").should be_false end end - it "does not scope a method body just because its call is inside a tenant block" do + it "finds raw connection SQL naming a row tenant table but ignores model raw SQL" do with_tempdir do |root| - write_rule_pack_project(root) - Dir.mkdir_p(File.join(root, "src", "models")) - File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") - file_path = File.join(root, "src", "jobs", "cleanup_job.cr") - Dir.mkdir_p(File.dirname(file_path)) - content = <<-CRYSTAL - def load_todos - Todo.all + install_tenancy_fixture_app(root, "row_app") + + connection_sql = <<-CRYSTAL + Invoice.adapter.open do |db| + db.exec("SELECT * FROM invoices WHERE account_id = 7") end - Grant::Tenant.with(7) { load_todos } CRYSTAL + connection_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/raw_report_job.cr", connection_sql) + has_tenancy_diagnostic_code?(connection_diagnostics, "grant/raw-connection-sql-on-tenant-table").should be_true - diagnostics = analyze_pack_file(root, file_path, content) + model_raw_sql = <<-CRYSTAL + Invoice.unscoped do + Invoice.exec("SELECT * FROM invoices") + end + CRYSTAL + model_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/raw_report_job.cr", model_raw_sql) + has_tenancy_diagnostic_code?(model_diagnostics, "grant/raw-connection-sql-on-tenant-table").should be_false - diagnostics.map(&.code).should contain("grant/row-query-outside-tenant") + other_table_sql = <<-CRYSTAL + Invoice.adapter.open do |db| + db.exec("SELECT * FROM audit_events") + end + CRYSTAL + other_table_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/raw_report_job.cr", other_table_sql) + has_tenancy_diagnostic_code?(other_table_diagnostics, "grant/raw-connection-sql-on-tenant-table").should be_false + + default_table_model = File.read(File.join( + Dir.current, + "spec", + "fixtures", + "rule_pack_apps", + "row_app", + "src", + "models", + "ledger_entry.cr", + )) + analyze_tenancy_fixture_source(root, "src/models/ledger_entry.cr", default_table_model) + default_table_sql = <<-CRYSTAL + Grant::Connections["primary"][:writer].open do |db| + db.scalar("SELECT COUNT(*) FROM ledger_entry") + end + CRYSTAL + default_table_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/raw_report_job.cr", default_table_sql) + has_tenancy_diagnostic_code?(default_table_diagnostics, "grant/raw-connection-sql-on-tenant-table").should be_true + + annotated_table_model = File.read(File.join( + Dir.current, + "spec", + "fixtures", + "rule_pack_apps", + "row_app", + "src", + "models", + "custom_document.cr", + )) + analyze_tenancy_fixture_source(root, "src/models/custom_document.cr", annotated_table_model) + annotated_table_sql = <<-CRYSTAL + Grant::Connections["primary"][:writer].open do |db| + db.query("SELECT * FROM custom_documents") { } + end + CRYSTAL + annotated_table_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/raw_report_job.cr", annotated_table_sql) + has_tenancy_diagnostic_code?(annotated_table_diagnostics, "grant/raw-connection-sql-on-tenant-table").should be_true end end - it "allows a same-model unscoped block but not a different model's query" do + it "warns about Tenant.clear outside specs and allows it in specs" do with_tempdir do |root| - write_rule_pack_project(root) - Dir.mkdir_p(File.join(root, "src", "models")) - File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") - File.write(File.join(root, "src", "models", "invoice.cr"), "class Invoice\n multitenant :tenant_id\nend\n") - file_path = File.join(root, "src", "jobs", "cleanup_job.cr") - Dir.mkdir_p(File.dirname(file_path)) - - same_model = analyze_pack_file(root, file_path, "Todo.unscoped { Todo.where(active: true) }\n") - same_model.map(&.code).should_not contain("grant/row-query-outside-tenant") - - different_model = analyze_pack_file(root, file_path, "Todo.unscoped { Invoice.all }\n") - different_model.map(&.code).should contain("grant/row-query-outside-tenant") + install_tenancy_fixture_app(root, "row_app") + + app_diagnostics = analyze_tenancy_fixture_source( + root, + "src/jobs/reset_tenant_job.cr", + "Grant::Tenant.clear\n", + ) + has_tenancy_diagnostic_code?(app_diagnostics, "grant/tenant-clear-in-app-code").should be_true + + spec_diagnostics = analyze_tenancy_fixture_source( + root, + "spec/reset_tenant_spec.cr", + "Grant::Tenant.clear\n", + ) + has_tenancy_diagnostic_code?(spec_diagnostics, "grant/tenant-clear-in-app-code").should be_false + + other_clear = analyze_tenancy_fixture_source(root, "src/jobs/reset_tenant_job.cr", "Grant::SchemaTenant.clear\n") + has_tenancy_diagnostic_code?(other_clear, "grant/tenant-clear-in-app-code").should be_false end end - it "does not flag raw_all and allows raw SQL inside the same model's unscoped block" do + it "warns about schema queries outside a tenant block and skips excluded models" do with_tempdir do |root| - write_rule_pack_project(root) - Dir.mkdir_p(File.join(root, "src", "models")) - File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") - file_path = File.join(root, "src", "jobs", "cleanup_job.cr") - Dir.mkdir_p(File.dirname(file_path)) - - raw_all_diagnostics = analyze_pack_file(root, file_path, "Todo.raw_all(\"WHERE active = true\")\n") - raw_all_diagnostics.map(&.code).should_not contain("grant/raw-sql-on-scoped-model") - - scoped_sql = <<-CRYSTAL - Todo.unscoped do - Todo.exec("DELETE FROM todos") - Todo.query("SELECT 1") { } - Todo.scalar("SELECT COUNT(*) FROM todos") { |value| value } - end - CRYSTAL - scoped_sql_diagnostics = analyze_pack_file(root, file_path, scoped_sql) - scoped_sql_diagnostics.map(&.code).should_not contain("grant/raw-sql-on-scoped-model") + install_tenancy_fixture_app(root, "schema_app") + + unsafe_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/rebuild_invoice_job.cr", "Invoice.all\n") + has_tenancy_diagnostic_code?(unsafe_diagnostics, "grant/schema-query-outside-tenant").should be_true + + scoped_diagnostics = analyze_tenancy_fixture_source( + root, + "src/jobs/rebuild_invoice_job.cr", + "Grant::SchemaTenant.with(\"acme\") { Invoice.all }\n", + ) + has_tenancy_diagnostic_code?(scoped_diagnostics, "grant/schema-query-outside-tenant").should be_false + + excluded_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/rebuild_invoice_job.cr", "Country.all\n") + has_tenancy_diagnostic_code?(excluded_diagnostics, "grant/schema-query-outside-tenant").should be_false - unsafe_sql = "Todo.exec(\"DELETE FROM todos\")\nTodo.query(\"SELECT 1\") { }\nTodo.scalar(\"SELECT 1\") { |value| value }\n" - unsafe_sql_diagnostics = analyze_pack_file(root, file_path, unsafe_sql) - unsafe_sql_diagnostics.count(&.code.==("grant/raw-sql-on-scoped-model")).should eq(3) + request_diagnostics = analyze_tenancy_fixture_source( + root, + "src/controllers/invoices_controller.cr", + "Invoice.all\n", + ) + has_tenancy_diagnostic_code?(request_diagnostics, "grant/schema-query-outside-tenant").should be_false end end - it "reports mixed modes as an error and undeclared row use as a warning" do + it "detects schema mode from schema_tenant_excluded without requiring a with call" do with_tempdir do |root| - write_rule_pack_project(root) - file_path = File.join(root, "src", "jobs", "tenancy_job.cr") - Dir.mkdir_p(File.dirname(file_path)) - mixed_content = "Grant::SchemaTenant.with(\"acme\") { run_job }\n" - - mixed_diagnostics = analyze_pack_file(root, file_path, mixed_content) - mixed_diagnostic = mixed_diagnostics.find(&.code.==("grant/tenancy-modes-mixed")).not_nil! - mixed_diagnostic.severity.should eq(AmberLSP::Rules::Severity::Error) - - shard_content = "name: tenant_app\nversion: 0.1.0\n" - write_rule_pack_project(root, shard_content) - Dir.mkdir_p(File.join(root, "src", "models")) - File.write(File.join(root, "src", "models", "todo.cr"), "class Todo\n multitenant :tenant_id\nend\n") - undeclared_path = File.join(root, "src", "jobs", "undeclared_job.cr") - undeclared_diagnostics = analyze_pack_file(root, undeclared_path, "Todo.all\n") - undeclared = undeclared_diagnostics.find(&.code.==("grant/tenancy-undeclared")).not_nil! - undeclared.severity.should eq(AmberLSP::Rules::Severity::Warning) + install_tenancy_fixture_app(root, "schema_app") + File.delete(File.join(root, "config", "tenant_scope.cr")) + + diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/rebuild_invoice_job.cr", "Invoice.all\n") + + has_tenancy_diagnostic_code?(diagnostics, "grant/schema-query-outside-tenant").should be_true end end - it "keeps Amber built-in rules gated on Amber dependencies" do + it "warns when both source-detected modes appear and leaves either mode alone" do with_tempdir do |root| - write_rule_pack_project(root) - file_path = File.join(root, "src", "controllers", "home_controller.cr") - Dir.mkdir_p(File.dirname(file_path)) - content = "class HomeController < ApplicationController\n def index\n User.all\n end\nend\n" - AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::Controllers::ActionReturnRule.new) - non_amber_analyzer = AmberLSP::Analyzer.new - non_amber_analyzer.configure(AmberLSP::ProjectContext.detect(root)) - - non_amber_diagnostics = non_amber_analyzer.analyze(file_path, content) - non_amber_diagnostics.map(&.code).should_not contain("amber/action-return-type") - - shard_content = <<-YAML - name: tenant_app - version: 0.1.0 - grant: - tenancy: row - dependencies: - amber: - github: amberframework/amber - YAML - File.write(File.join(root, "shard.yml"), shard_content) - amber_analyzer = AmberLSP::Analyzer.new - amber_analyzer.configure(AmberLSP::ProjectContext.detect(root)) - - amber_diagnostics = amber_analyzer.analyze(file_path, content) - amber_diagnostics.map(&.code).should contain("amber/action-return-type") + install_tenancy_fixture_app(root, "mixed_app") + + mixed_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/check_tenant_job.cr", "Invoice.all\n") + has_tenancy_diagnostic_code?(mixed_diagnostics, "grant/tenancy-modes-mixed").should be_true + mixed_diagnostics.any? do |diagnostic| + diagnostic.code == "grant/tenancy-modes-mixed" && diagnostic.message.includes?("docs/schema_tenancy.md") + end.should be_true + + install_tenancy_fixture_app(root, "row_app") + row_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/check_tenant_job.cr", "Invoice.all\n") + has_tenancy_diagnostic_code?(row_diagnostics, "grant/tenancy-modes-mixed").should be_false + + install_tenancy_fixture_app(root, "schema_app") + schema_diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/check_tenant_job.cr", "Invoice.all\n") + has_tenancy_diagnostic_code?(schema_diagnostics, "grant/tenancy-modes-mixed").should be_false end end - it "prints declared context and warns when feature use has no declaration" do + it "ignores tenancy keys in shard.yml and reports no diagnostics for an app with no tenancy" do with_tempdir do |root| - write_rule_pack_project(root) - file_path = File.join(root, "src", "models", "todo.cr") - Dir.mkdir_p(File.dirname(file_path)) - File.write(file_path, "class Todo\n multitenant :tenant_id\nend\n") - binary_path = File.join(Dir.current, "bin", "amber-lsp") - stdout = IO::Memory.new - stderr = IO::Memory.new - status = Process.run(binary_path, ["context", "--root", root], output: stdout, error: stderr) - - status.success?.should be_true - stderr.to_s.should be_empty - stdout.to_s.should contain("grant/tenancy (row)") - stdout.to_s.should contain("Scope queries with Grant::Tenant.with.") - - undeclared_shard = "name: tenant_app\nversion: 0.1.0\n" - write_rule_pack_project(root, undeclared_shard) - stdout = IO::Memory.new - stderr = IO::Memory.new - status = Process.run(binary_path, ["context", "--root", root], output: stdout, error: stderr) - - status.success?.should be_true - stdout.to_s.should contain("warning: grant/tenancy feature is used but shard.yml does not declare grant.tenancy.") + install_tenancy_fixture_app(root, "no_tenancy_app") + File.write( + File.join(root, "shard.yml"), + "name: no_tenancy_app\nversion: 0.1.0\ngrant:\n tenancy: row\n", + ) + Dir.mkdir_p(File.join(root, "spec")) + File.write(File.join(root, "spec", "fake_tenancy.cr"), "Grant::SchemaTenant.with(\"spec\") { nil }\n") + Dir.mkdir_p(File.join(root, "lib", "other")) + File.write(File.join(root, "lib", "other", "fake_tenancy.cr"), "Grant::SchemaTenant.with(\"lib\") { nil }\n") + File.write( + File.join(root, "src", "models", "commented_macro.cr"), + "# multitenant :account_id\nMESSAGE = \"schema_tenant_excluded\"\nclass Invoice\n def configure\n multitenant :account_id\n end\nend\n", + ) + + diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/clean_job.cr", "Grant::Tenant.clear\nInvoice.all\n") + + diagnostics.should be_empty + File.read(File.join(root, "shard.yml")).should contain("grant:") end end - it "prints nothing when no pack is declared or evidenced" do + it "skips malformed Crystal files without crashing or inventing another mode" do with_tempdir do |root| - File.write(File.join(root, "shard.yml"), "name: empty_app\nversion: 0.1.0\n") - binary_path = File.join(Dir.current, "bin", "amber-lsp") - stdout = IO::Memory.new - status = Process.run(binary_path, ["context", "--root", root], output: stdout, error: Process::Redirect::Close) + install_tenancy_fixture_app(root, "row_app") + Dir.mkdir_p(File.join(root, "config")) + File.write(File.join(root, "config", "broken.cr"), "Grant::SchemaTenant.with(\"broken\") do\n") + + diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/clean_job.cr", "Invoice.all\n") - status.success?.should be_true - stdout.to_s.should be_empty + diagnostics.should be_empty end end end diff --git a/spec/amber_lsp/spec_helper.cr b/spec/amber_lsp/spec_helper.cr index 822b2f0..6a2a665 100644 --- a/spec/amber_lsp/spec_helper.cr +++ b/spec/amber_lsp/spec_helper.cr @@ -12,9 +12,17 @@ require "../../src/amber_lsp/configuration" require "../../src/amber_lsp/library_rule_packs/describe_library_rule_pack" require "../../src/amber_lsp/library_rule_packs/load_rule_packs_for_project" require "../../src/amber_lsp/library_rule_packs/visit_crystal_calls_outside_required_blocks" +require "../../src/amber_lsp/library_rule_packs/grant_tenancy/source_node" +require "../../src/amber_lsp/library_rule_packs/grant_tenancy/grant_tenant_model_declaration" +require "../../src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations" require "../../src/amber_lsp/library_rule_packs/determine_project_rule_pack_state" +require "../../src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls" +require "../../src/amber_lsp/library_rule_packs/grant_tenancy/visit_spawn_calls_inside_grant_tenant_blocks" +require "../../src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_tenant_clear_calls" +require "../../src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites" +require "../../src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_schema_queries_outside_tenant_blocks" require "../../src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs" -require "../../src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts" +require "../../src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts" require "../../src/amber_lsp/analyzer" require "../../src/amber_lsp/controller" require "../../src/amber_lsp/server" diff --git a/spec/fixtures/rule_pack_apps/mixed_app/config/tenant_scope.cr b/spec/fixtures/rule_pack_apps/mixed_app/config/tenant_scope.cr new file mode 100644 index 0000000..6568adb --- /dev/null +++ b/spec/fixtures/rule_pack_apps/mixed_app/config/tenant_scope.cr @@ -0,0 +1,3 @@ +Grant::SchemaTenant.with("acme") do + Invoice.all +end diff --git a/spec/fixtures/rule_pack_apps/mixed_app/src/models/invoice.cr b/spec/fixtures/rule_pack_apps/mixed_app/src/models/invoice.cr new file mode 100644 index 0000000..54a057d --- /dev/null +++ b/spec/fixtures/rule_pack_apps/mixed_app/src/models/invoice.cr @@ -0,0 +1,8 @@ +class Invoice < Grant::Base + table :invoices + + column id : Int64, primary: true + column account_id : Int64 + + multitenant :account_id +end diff --git a/spec/fixtures/rule_pack_apps/no_tenancy_app/src/models/invoice.cr b/spec/fixtures/rule_pack_apps/no_tenancy_app/src/models/invoice.cr new file mode 100644 index 0000000..95839d3 --- /dev/null +++ b/spec/fixtures/rule_pack_apps/no_tenancy_app/src/models/invoice.cr @@ -0,0 +1,6 @@ +class Invoice < Grant::Base + table :invoices + + column id : Int64, primary: true + column account_id : Int64 +end diff --git a/spec/fixtures/rule_pack_apps/row_app/lib/grant/.amber-lsp/packs/tenancy.yml b/spec/fixtures/rule_pack_apps/row_app/lib/grant/.amber-lsp/packs/tenancy.yml new file mode 100644 index 0000000..48a0431 --- /dev/null +++ b/spec/fixtures/rule_pack_apps/row_app/lib/grant/.amber-lsp/packs/tenancy.yml @@ -0,0 +1,142 @@ +pack: grant/tenancy +library: grant +version: 2.0.0 + +modes: + row: + context: | + Grant row tenancy was detected from the app's `multitenant` model macros. + Each model captures the tenant column passed to its own macro. Grant adds a + default scope that raises `Grant::NoTenantError` when no tenant is set, and + writes using another tenant's id raise `Grant::TenantMismatchError`. + `Model.exec/query/scalar` on a default-scoped model raises + `Grant::Querying::ScopedRawSqlError` outside `Model.unscoped { }`. + Block-form `unscoped` temporarily disables scoping. Chainable `Model.unscoped` + returns an unguarded builder, so chained reads and writes can cross tenants. + `Grant::Tenant.with` is fiber-local: a spawned fiber starts without the + tenant. Carry the tenant id into that fiber and wrap its work. `Tenant.clear` + exists mainly for tests. + schema: + context: | + Grant schema tenancy was detected from the app's `Grant::SchemaTenant.with` + call or `schema_tenant_excluded` model macro. `SchemaTenant.with` pins a + PostgreSQL connection with the tenant search path; excluded models use + `public`. A query on a non-excluded model outside the block runs against the + default search path without an error. The request pipe covers controllers; + jobs and other non-request work need their own block. The row and schema + modes can coexist, but review `docs/schema_tenancy.md` for their tradeoffs + and migration path. + +rules: + - id: grant/chained-unscoped-in-request-code + modes: [row] + severity: error + applies_to: + - "src/controllers/**" + - "src/channels/**" + - "src/sockets/**" + - "src/pipes/**" + message: "Remove chainable unscoped from request code and rely on the model's tenant scope." + check: + kind: crystal_ast + operation: chained_unscoped_in_request_code + + - id: grant/chained-unscoped-bulk-write + modes: [row] + severity: error + applies_to: ["**/*.cr"] + exclude_from: ["spec/**", "db/**"] + message: "Replace the unscoped bulk write with a tenant-scoped write or an explicit tenant predicate." + check: + kind: crystal_ast + operation: chained_unscoped_bulk_write + + - id: grant/chained-unscoped-on-tenant-model + modes: [row] + severity: warning + applies_to: ["**/*.cr"] + exclude_from: + - "src/controllers/**" + - "src/channels/**" + - "src/sockets/**" + - "src/pipes/**" + - "spec/**" + - "db/**" + message: "Use the tenant-scoped model query; reserve chainable unscoped for an explicitly justified cross-tenant operation." + check: + kind: crystal_ast + operation: chained_unscoped_on_tenant_model + + - id: grant/unscoped-block-in-request-code + modes: [row] + severity: warning + applies_to: + - "src/controllers/**" + - "src/channels/**" + - "src/sockets/**" + - "src/pipes/**" + message: "Remove the unscoped block from request code and rely on the model's tenant scope." + check: + kind: crystal_ast + operation: unscoped_block_in_request_code + + - id: grant/spawn-inside-tenant-block + modes: [row, schema] + severity: warning + applies_to: ["**/*.cr"] + exclude_from: ["spec/**"] + message: "Carry the tenant id or schema into the fiber and wrap its work in the matching Grant tenant block." + check: + kind: crystal_ast + operation: spawn_inside_tenant_block + + - id: grant/tenant-column-without-multitenant + modes: [row] + severity: warning + applies_to: ["src/**"] + message: "This model declares {tenant_column}, which is used as a tenant key elsewhere; add multitenant :{tenant_column}, or rename/remove it if unrelated." + check: + kind: crystal_ast + operation: tenant_column_without_multitenant + + - id: grant/raw-connection-sql-on-tenant-table + modes: [row] + severity: warning + applies_to: ["**/*.cr"] + exclude_from: ["spec/**"] + message: "Raw connection SQL bypasses model scopes; use the tenant-scoped model API or add an explicit tenant predicate." + check: + kind: crystal_ast + operation: raw_connection_sql_on_tenant_table + + - id: grant/tenant-clear-in-app-code + modes: [row] + severity: warning + applies_to: ["**/*.cr"] + exclude_from: ["spec/**"] + message: "Remove Tenant.clear from app code; scope work with Grant::Tenant.with and reserve clear for tests." + check: + kind: crystal_ast + operation: tenant_clear_in_app_code + + - id: grant/schema-query-outside-tenant + modes: [schema] + severity: warning + applies_to: + - "src/jobs/**" + - "src/lib/**" + - "src/domain/**" + - "src/process_managers/**" + message: "Wrap non-request work in Grant::SchemaTenant.with(schema); otherwise this query uses the default search path." + check: + kind: crystal_ast + operation: schema_query_outside_tenant + + - id: grant/tenancy-modes-mixed + modes: [row, schema] + severity: warning + applies_to: ["**/*.cr"] + message: "Both tenancy modes were detected; review docs/schema_tenancy.md and document why both are needed." + check: + kind: project_conflict + condition: mixed_modes diff --git a/spec/fixtures/rule_pack_apps/row_app/src/models/account.cr b/spec/fixtures/rule_pack_apps/row_app/src/models/account.cr new file mode 100644 index 0000000..3d8f17b --- /dev/null +++ b/spec/fixtures/rule_pack_apps/row_app/src/models/account.cr @@ -0,0 +1,6 @@ +class Account < Grant::Base + table :accounts + + column id : Int64, primary: true + column account_id : Int64 +end diff --git a/spec/fixtures/rule_pack_apps/row_app/src/models/custom_document.cr b/spec/fixtures/rule_pack_apps/row_app/src/models/custom_document.cr new file mode 100644 index 0000000..d09e19f --- /dev/null +++ b/spec/fixtures/rule_pack_apps/row_app/src/models/custom_document.cr @@ -0,0 +1,5 @@ +@[Grant::Table(name: :custom_documents)] +class CustomDocument < Grant::Base + column account_id : Int64 + multitenant :account_id +end diff --git a/spec/fixtures/rule_pack_apps/row_app/src/models/invoice.cr b/spec/fixtures/rule_pack_apps/row_app/src/models/invoice.cr new file mode 100644 index 0000000..54a057d --- /dev/null +++ b/spec/fixtures/rule_pack_apps/row_app/src/models/invoice.cr @@ -0,0 +1,8 @@ +class Invoice < Grant::Base + table :invoices + + column id : Int64, primary: true + column account_id : Int64 + + multitenant :account_id +end diff --git a/spec/fixtures/rule_pack_apps/row_app/src/models/invoice_export.cr b/spec/fixtures/rule_pack_apps/row_app/src/models/invoice_export.cr new file mode 100644 index 0000000..c49b0a7 --- /dev/null +++ b/spec/fixtures/rule_pack_apps/row_app/src/models/invoice_export.cr @@ -0,0 +1,4 @@ +class InvoiceExport < Grant::Base + table :invoice_exports + column account_id : Int64 +end diff --git a/spec/fixtures/rule_pack_apps/row_app/src/models/ledger_entry.cr b/spec/fixtures/rule_pack_apps/row_app/src/models/ledger_entry.cr new file mode 100644 index 0000000..eece661 --- /dev/null +++ b/spec/fixtures/rule_pack_apps/row_app/src/models/ledger_entry.cr @@ -0,0 +1,5 @@ +class LedgerEntry < Grant::Base + column id : Int64, primary: true + column account_id : Int64 + multitenant :account_id +end diff --git a/spec/fixtures/rule_pack_apps/schema_app/config/tenant_scope.cr b/spec/fixtures/rule_pack_apps/schema_app/config/tenant_scope.cr new file mode 100644 index 0000000..6568adb --- /dev/null +++ b/spec/fixtures/rule_pack_apps/schema_app/config/tenant_scope.cr @@ -0,0 +1,3 @@ +Grant::SchemaTenant.with("acme") do + Invoice.all +end diff --git a/spec/fixtures/rule_pack_apps/schema_app/src/models/country.cr b/spec/fixtures/rule_pack_apps/schema_app/src/models/country.cr new file mode 100644 index 0000000..8180127 --- /dev/null +++ b/spec/fixtures/rule_pack_apps/schema_app/src/models/country.cr @@ -0,0 +1,7 @@ +class Country < Grant::Base + table :countries + + column id : Int64, primary: true + + schema_tenant_excluded +end diff --git a/spec/fixtures/rule_pack_apps/schema_app/src/models/invoice.cr b/spec/fixtures/rule_pack_apps/schema_app/src/models/invoice.cr new file mode 100644 index 0000000..b235c74 --- /dev/null +++ b/spec/fixtures/rule_pack_apps/schema_app/src/models/invoice.cr @@ -0,0 +1,5 @@ +class Invoice < Grant::Base + table :invoices + + column id : Int64, primary: true +end diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index d4ae72e..85d511f 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -24,15 +24,23 @@ require "./amber_lsp/configuration" require "./amber_lsp/library_rule_packs/describe_library_rule_pack" require "./amber_lsp/library_rule_packs/load_rule_packs_for_project" require "./amber_lsp/library_rule_packs/visit_crystal_calls_outside_required_blocks" +require "./amber_lsp/library_rule_packs/grant_tenancy/source_node" +require "./amber_lsp/library_rule_packs/grant_tenancy/grant_tenant_model_declaration" +require "./amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations" require "./amber_lsp/library_rule_packs/determine_project_rule_pack_state" +require "./amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls" +require "./amber_lsp/library_rule_packs/grant_tenancy/visit_spawn_calls_inside_grant_tenant_blocks" +require "./amber_lsp/library_rule_packs/grant_tenancy/visit_grant_tenant_clear_calls" +require "./amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites" +require "./amber_lsp/library_rule_packs/grant_tenancy/visit_grant_schema_queries_outside_tenant_blocks" require "./amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs" -require "./amber_lsp/library_rule_packs/print_declared_rule_pack_contexts" +require "./amber_lsp/library_rule_packs/print_detected_rule_pack_contexts" require "./amber_lsp/analyzer" require "./amber_lsp/controller" require "./amber_lsp/server" if ARGV.first? == "context" - exit AmberLSP::LibraryRulePacks::PrintDeclaredRulePackContexts.new(ARGV[1..].to_a).perform + exit AmberLSP::LibraryRulePacks::PrintDetectedRulePackContexts.new(ARGV[1..].to_a).perform else AmberLSP::Server.new(STDIN, STDOUT).run end diff --git a/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr b/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr index da0595d..109a347 100644 --- a/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr +++ b/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr @@ -36,15 +36,15 @@ module AmberLSP::LibraryRulePacks if rule.check.check_kind == "project_conflict" list_of_diagnostics.concat( - project_conflict_diagnostics_for(rule_pack, rule, project_state) + project_conflict_diagnostics_for(rule, project_state) ) next end - next unless project_state.is_rule_mode_declared?(rule.list_of_mode_names) + next unless project_state.rule_mode_detected?(rule.list_of_mode_names) list_of_diagnostics.concat( - file_diagnostics_for(rule_pack, rule, project_state, file_path, relative_file_path, content) + file_diagnostics_for(rule, project_state, file_path, relative_file_path, content) ) end end @@ -68,15 +68,12 @@ module AmberLSP::LibraryRulePacks end private def project_conflict_diagnostics_for( - rule_pack : DescribeLibraryRulePack, rule : DescribeLibraryRulePack::Rule, project_state : DetermineProjectRulePackState, ) : Array(Rules::Diagnostic) conflict_found = case rule.check.project_condition when "mixed_modes" project_state.all_modes_are_present?(rule.list_of_mode_names) - when "evidence_without_declaration" - project_state.has_undeclared_mode_evidence?(rule.list_of_mode_names) else false end @@ -86,7 +83,6 @@ module AmberLSP::LibraryRulePacks end private def file_diagnostics_for( - rule_pack : DescribeLibraryRulePack, rule : DescribeLibraryRulePack::Rule, project_state : DetermineProjectRulePackState, file_path : String, @@ -97,9 +93,11 @@ module AmberLSP::LibraryRulePacks when "line_regex" line_regex_diagnostics_for(rule, relative_file_path, content) when "file_requires" - file_requires_diagnostics_for(rule_pack, rule, content) + file_requires_diagnostics_for(rule, content) when "call_outside_block" call_outside_block_diagnostics_for(rule, project_state, file_path, content) + when "crystal_ast" + crystal_ast_diagnostics_for(rule, project_state, file_path, content) else [] of Rules::Diagnostic end @@ -123,14 +121,13 @@ module AmberLSP::LibraryRulePacks end private def file_requires_diagnostics_for( - rule_pack : DescribeLibraryRulePack, rule : DescribeLibraryRulePack::Rule, content : String, ) : Array(Rules::Diagnostic) required_pattern = Regex.new(rule.check.required_regex_pattern) return [] of Rules::Diagnostic if content.each_line.any? { |line| required_pattern.matches?(line) } - trigger_pattern = trigger_pattern_for(rule_pack, rule) + trigger_pattern = trigger_pattern_for(rule) return [] of Rules::Diagnostic unless trigger_pattern list_of_diagnostics = [] of Rules::Diagnostic @@ -155,20 +152,12 @@ module AmberLSP::LibraryRulePacks list_of_diagnostics end - private def trigger_pattern_for( - rule_pack : DescribeLibraryRulePack, - rule : DescribeLibraryRulePack::Rule, - ) : Regex? + private def trigger_pattern_for(rule : DescribeLibraryRulePack::Rule) : Regex? unless rule.check.trigger_regex_pattern.empty? return Regex.new(rule.check.trigger_regex_pattern) end - mode = rule_pack.modes_by_name[rule.list_of_mode_names.first]? - tenant_column_name = mode.try(&.tenant_column_name) - return nil unless tenant_column_name - return nil unless tenant_column_name.matches?(/\A[a-zA-Z_][a-zA-Z0-9_]*\z/) - - Regex.new("^\\s*column\\s+#{tenant_column_name}\\b") + nil end private def call_outside_block_diagnostics_for( @@ -215,6 +204,112 @@ module AmberLSP::LibraryRulePacks [] of Rules::Diagnostic end + private def crystal_ast_diagnostics_for( + rule : DescribeLibraryRulePack::Rule, + project_state : DetermineProjectRulePackState, + file_path : String, + content : String, + ) : Array(Rules::Diagnostic) + ast = Crystal::Parser.new(content).parse + + case rule.check.operation_name + when "chained_unscoped_in_request_code" + visitor = GrantTenancy::VisitChainableUnscopedModelCalls.new(project_state) + visitor.accept(ast) + diagnostics_for_unscoped_calls(visitor.list_of_chainable_unscoped_calls, rule) + when "chained_unscoped_bulk_write" + visitor = GrantTenancy::VisitChainableUnscopedModelCalls.new(project_state) + visitor.accept(ast) + list_of_bulk_write_calls = visitor.list_of_chainable_unscoped_calls.select(&.has_bulk_write_after?) + diagnostics_for_unscoped_calls(list_of_bulk_write_calls, rule) + when "chained_unscoped_on_tenant_model" + visitor = GrantTenancy::VisitChainableUnscopedModelCalls.new(project_state) + visitor.accept(ast) + list_of_read_calls = visitor.list_of_chainable_unscoped_calls.reject(&.has_bulk_write_after?) + diagnostics_for_unscoped_calls(list_of_read_calls, rule) + when "unscoped_block_in_request_code" + visitor = GrantTenancy::VisitChainableUnscopedModelCalls.new(project_state) + visitor.accept(ast) + diagnostics_for_calls(visitor.list_of_block_unscoped_calls, rule) + when "spawn_inside_tenant_block" + visitor = GrantTenancy::VisitSpawnCallsInsideGrantTenantBlocks.new + visitor.accept(ast) + diagnostics_for_calls(visitor.list_of_spawn_calls_inside_tenant_blocks, rule) + when "tenant_column_without_multitenant" + list_of_findings = project_state.list_of_tenant_column_declarations_for(file_path) + diagnostics_for_tenant_column_findings(list_of_findings, rule) + when "raw_connection_sql_on_tenant_table" + visitor = GrantTenancy::VisitRawConnectionSqlCallSites.new(project_state) + visitor.accept(ast) + list_of_calls = visitor.list_of_raw_connection_sql_call_sites.map(&.call) + diagnostics_for_calls(list_of_calls, rule) + when "tenant_clear_in_app_code" + visitor = GrantTenancy::VisitGrantTenantClearCalls.new + visitor.accept(ast) + diagnostics_for_calls(visitor.list_of_tenant_clear_calls, rule) + when "schema_query_outside_tenant" + visitor = GrantTenancy::VisitGrantSchemaQueriesOutsideTenantBlocks.new(project_state) + visitor.accept(ast) + diagnostics_for_calls(visitor.list_of_schema_queries_outside_tenant_blocks, rule) + else + [] of Rules::Diagnostic + end + rescue Crystal::SyntaxException + [] of Rules::Diagnostic + end + + private def diagnostics_for_unscoped_calls( + list_of_occurrences : Array(GrantTenancy::VisitChainableUnscopedModelCalls::Occurrence), + rule : DescribeLibraryRulePack::Rule, + ) : Array(Rules::Diagnostic) + list_of_occurrences.compact_map { |occurrence| diagnostic_for_source_node(rule, occurrence.call) } + end + + private def diagnostics_for_tenant_column_findings( + list_of_findings : Array(Tuple(String, Crystal::ASTNode)), + rule : DescribeLibraryRulePack::Rule, + ) : Array(Rules::Diagnostic) + list_of_findings.compact_map do |finding| + diagnostic_for_source_node(rule, finding[1], finding[0]) + end + end + + private def diagnostics_for_calls( + list_of_calls : Array(Crystal::Call), + rule : DescribeLibraryRulePack::Rule, + ) : Array(Rules::Diagnostic) + list_of_calls.compact_map { |call| diagnostic_for_source_node(rule, call) } + end + + private def diagnostic_for_source_node( + rule : DescribeLibraryRulePack::Rule, + source_node : Crystal::ASTNode, + source_name : String? = nil, + ) : Rules::Diagnostic? + location = if source_node.is_a?(Crystal::Call) + source_node.name_location || source_node.location + else + source_node.location + end + return nil unless location + + start_character = (location.column_number - 1).to_i32 + display_name = source_name || source_node.as?(Crystal::Call).try(&.name) || "" + end_character = start_character + display_name.size + range = Rules::TextRange.new( + Rules::Position.new((location.line_number - 1).to_i32, start_character), + Rules::Position.new((location.line_number - 1).to_i32, end_character), + ) + diagnostic_message = source_name ? rule.diagnostic_message.gsub("{tenant_column}", source_name) : rule.diagnostic_message + + Rules::Diagnostic.new( + range, + severity_for(rule.severity_name), + rule.rule_id, + diagnostic_message, + ) + end + private def diagnostic_at_start_of_file( rule : DescribeLibraryRulePack::Rule, severity : Rules::Severity, diff --git a/src/amber_lsp/library_rule_packs/describe_library_rule_pack.cr b/src/amber_lsp/library_rule_packs/describe_library_rule_pack.cr index 7d1a550..3621bc9 100644 --- a/src/amber_lsp/library_rule_packs/describe_library_rule_pack.cr +++ b/src/amber_lsp/library_rule_packs/describe_library_rule_pack.cr @@ -32,25 +32,9 @@ module AmberLSP::LibraryRulePacks end end - class Declaration - include YAML::Serializable - - property key_path : String - property expected_value : String - end - class Mode include YAML::Serializable - @[YAML::Field(key: "declared_by")] - property declaration : Declaration - - @[YAML::Field(key: "evidence")] - property list_of_evidence_patterns : Array(String) = [] of String - - @[YAML::Field(key: "tenant_column")] - property tenant_column_name : String? = nil - @[YAML::Field(key: "context")] property guidance_text : String = "" end @@ -114,6 +98,9 @@ module AmberLSP::LibraryRulePacks @[YAML::Field(key: "condition")] property project_condition : String = "mixed_modes" + + @[YAML::Field(key: "operation")] + property operation_name : String = "" end end end diff --git a/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr b/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr index dd7ccc8..0ce69af 100644 --- a/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr +++ b/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr @@ -2,9 +2,12 @@ require "set" module AmberLSP::LibraryRulePacks class DetermineProjectRulePackState - @is_mode_declared_by_name = {} of String => Bool - @has_mode_evidence_by_name = {} of String => Bool + @uses_row_tenancy = false + @uses_schema_tenancy = false @project_source_content_by_path = {} of String => String + @list_of_model_declarations_by_file = {} of String => Array(GrantTenancy::GrantTenantModelDeclaration) + @list_of_model_declarations = [] of GrantTenancy::GrantTenantModelDeclaration + @list_of_row_tenant_column_names = Set(String).new def initialize( @project_context : AmberLSP::ProjectContext, @@ -13,40 +16,76 @@ module AmberLSP::LibraryRulePacks @file_content : String, ) load_project_source_content - determine_mode_declarations - determine_mode_evidence end - def is_mode_declared?(mode_name : String) : Bool - @is_mode_declared_by_name[mode_name]? || false + def mode_detected?(mode_name : String) : Bool + case mode_name + when "row" + @uses_row_tenancy + when "schema" + @uses_schema_tenancy + else + false + end end - def has_evidence_for_mode?(mode_name : String) : Bool - @has_mode_evidence_by_name[mode_name]? || false + def has_any_applicable_mode? : Bool + @rule_pack.modes_by_name.keys.any? { |mode_name| mode_detected?(mode_name) } end - def is_mode_present?(mode_name : String) : Bool - is_mode_declared?(mode_name) || has_evidence_for_mode?(mode_name) + def all_modes_are_present?(list_of_mode_names : Array(String)) : Bool + !list_of_mode_names.empty? && list_of_mode_names.all? { |mode_name| mode_detected?(mode_name) } end - def has_any_applicable_mode? : Bool - @rule_pack.modes_by_name.keys.any? do |mode_name| - is_mode_declared?(mode_name) || has_evidence_for_mode?(mode_name) - end + def rule_mode_detected?(list_of_mode_names : Array(String)) : Bool + list_of_mode_names.any? { |mode_name| mode_detected?(mode_name) } end - def all_modes_are_present?(list_of_mode_names : Array(String)) : Bool - !list_of_mode_names.empty? && list_of_mode_names.all? { |mode_name| is_mode_present?(mode_name) } + def has_row_tenant_model?(model_reference_name : String) : Bool + matching_model_declarations(model_reference_name).any?(&.has_multitenant_macro?) end - def has_undeclared_mode_evidence?(list_of_mode_names : Array(String)) : Bool - list_of_mode_names.any? do |mode_name| - has_evidence_for_mode?(mode_name) && !is_mode_declared?(mode_name) + def has_schema_excluded_model?(model_reference_name : String) : Bool + matching_model_declarations(model_reference_name).any?(&.schema_tenant_excluded?) + end + + def has_non_excluded_schema_model?(model_reference_name : String) : Bool + matching_model_declarations(model_reference_name).any? do |model_declaration| + model_declaration_is_grant_model?(model_declaration) && + !has_schema_excluded_model?(model_declaration.qualified_model_name) end end - def is_rule_mode_declared?(list_of_mode_names : Array(String)) : Bool - list_of_mode_names.any? { |mode_name| is_mode_declared?(mode_name) } + def list_of_row_tenant_table_names : Array(String) + @list_of_model_declarations.compact_map do |model_declaration| + next unless model_declaration.has_multitenant_macro? + + model_declaration.source_table_name + end.uniq + end + + def list_of_tenant_column_declarations_for(file_path : String) : Array(Tuple(String, Crystal::ASTNode)) + absolute_file_path = File.expand_path(file_path) + list_of_current_file_models = @list_of_model_declarations_by_file[absolute_file_path]? + return [] of Tuple(String, Crystal::ASTNode) unless list_of_current_file_models + return [] of Tuple(String, Crystal::ASTNode) if @list_of_row_tenant_column_names.empty? + + list_of_findings = [] of Tuple(String, Crystal::ASTNode) + + list_of_current_file_models.each do |model_declaration| + next unless model_declaration_is_grant_model?(model_declaration) + next if has_row_tenant_model?(model_declaration.qualified_model_name) + + model_declaration.list_of_column_declarations.each do |column_declaration| + tenant_column_name = column_declaration[0] + next unless @list_of_row_tenant_column_names.includes?(tenant_column_name) + next if model_table_is_referenced_by_column?(model_declaration, tenant_column_name) + + list_of_findings << column_declaration + end + end + + list_of_findings end def list_of_scoped_model_names_for( @@ -68,61 +107,110 @@ module AmberLSP::LibraryRulePacks end private def load_project_source_content : Nil - project_root = File.expand_path(@project_context.root_path) + project_source_file_paths.each do |file_path| + content = content_for(file_path) + next unless content - Dir.glob(File.join(project_root, "**", "*.cr")).each do |file_path| - next unless project_file_is_application_source?(file_path, project_root) - - content = if File.expand_path(file_path) == File.expand_path(@file_path) - @file_content - else - File.read(file_path) - end - @project_source_content_by_path[File.expand_path(file_path)] = content - rescue - next + absolute_file_path = File.expand_path(file_path) + @project_source_content_by_path[absolute_file_path] = content + collect_project_declarations_from(absolute_file_path, content) end current_file_path = File.expand_path(@file_path) - if project_file_is_application_source?(current_file_path, project_root) + if project_file_is_application_source?(current_file_path) && + !@project_source_content_by_path.has_key?(current_file_path) @project_source_content_by_path[current_file_path] = @file_content + collect_project_declarations_from(current_file_path, @file_content) + end + end + + private def project_source_file_paths : Array(String) + project_root = File.expand_path(@project_context.root_path) + list_of_source_file_paths = Dir.glob(File.join(project_root, "src", "**", "*.cr")) + list_of_source_file_paths.concat(Dir.glob(File.join(project_root, "config", "**", "*.cr"))) + list_of_source_file_paths.uniq.sort + end + + private def content_for(file_path : String) : String? + if File.expand_path(file_path) == File.expand_path(@file_path) + return @file_content end + + File.read(file_path) + rescue IO::Error + nil end - private def project_file_is_application_source?(file_path : String, project_root : String) : Bool + private def collect_project_declarations_from(file_path : String, content : String) : Nil + collector = GrantTenancy::CollectProjectGrantTenancyDeclarations.for_source(content) + @uses_row_tenancy ||= collector.uses_row_tenancy? + @uses_schema_tenancy ||= collector.uses_schema_tenancy? + @list_of_model_declarations_by_file[file_path] = collector.list_of_model_declarations + @list_of_model_declarations.concat(collector.list_of_model_declarations) + + collector.list_of_model_declarations.each do |model_declaration| + tenant_column_name = model_declaration.tenant_column_name + next unless model_declaration.has_multitenant_macro? && tenant_column_name + + @list_of_row_tenant_column_names.add(tenant_column_name) + end + end + + private def project_file_is_application_source?(file_path : String) : Bool + project_root = File.expand_path(@project_context.root_path) prefix = project_root.ends_with?(File::SEPARATOR) ? project_root : "#{project_root}#{File::SEPARATOR}" return false unless file_path.starts_with?(prefix) relative_path = file_path[prefix.size..] - return false if relative_path.starts_with?("lib/") - return false if relative_path.starts_with?(".git/") - return false if relative_path.starts_with?("tmp/") - return false if relative_path.starts_with?("vendor/") - - true + (relative_path.starts_with?("src/") || relative_path.starts_with?("config/")) && + file_path.ends_with?(".cr") end - private def determine_mode_declarations : Nil - @rule_pack.modes_by_name.each do |mode_name, mode| - @is_mode_declared_by_name[mode_name] = @project_context.has_shard_declaration?( - mode.declaration.key_path, - mode.declaration.expected_value, - ) + private def matching_model_declarations(model_reference_name : String) : Array(GrantTenancy::GrantTenantModelDeclaration) + list_of_exact_matches = @list_of_model_declarations.select do |model_declaration| + model_declaration.qualified_model_name == model_reference_name + end + return list_of_exact_matches unless list_of_exact_matches.empty? + return [] of GrantTenancy::GrantTenantModelDeclaration if model_reference_name.includes?("::") + + list_of_simple_name_matches = @list_of_model_declarations.select do |model_declaration| + model_declaration.class_name == model_reference_name end + unique_qualified_names = list_of_simple_name_matches.map(&.qualified_model_name).uniq + return [] of GrantTenancy::GrantTenantModelDeclaration unless unique_qualified_names.size == 1 + + list_of_simple_name_matches end - private def determine_mode_evidence : Nil - @rule_pack.modes_by_name.each do |mode_name, mode| - evidence_found = mode.list_of_evidence_patterns.any? do |pattern| - regex = Regex.new(pattern) - @project_source_content_by_path.values.any? do |content| - content.each_line.any? { |line| regex.matches?(line) } - end - end - @has_mode_evidence_by_name[mode_name] = evidence_found + private def model_declaration_is_grant_model?( + model_declaration : GrantTenancy::GrantTenantModelDeclaration, + list_of_visited_model_names : Set(String) = Set(String).new, + ) : Bool + return true if model_declaration.grant_model? + return false if list_of_visited_model_names.includes?(model_declaration.qualified_model_name) + + list_of_visited_model_names.add(model_declaration.qualified_model_name) + superclass_reference_name = model_declaration.superclass_reference_name + return false unless superclass_reference_name + return true if superclass_reference_name == "Grant::Base" + + matching_model_declarations(superclass_reference_name).any? do |parent_model_declaration| + model_declaration_is_grant_model?(parent_model_declaration, list_of_visited_model_names) end - rescue ArgumentError - @rule_pack.modes_by_name.each_key { |mode_name| @has_mode_evidence_by_name[mode_name] = false } + end + + private def model_table_is_referenced_by_column?( + model_declaration : GrantTenancy::GrantTenantModelDeclaration, + tenant_column_name : String, + ) : Bool + return false unless tenant_column_name.ends_with?("_id") + + referenced_table_base_name = tenant_column_name[0, tenant_column_name.size - 3] + list_of_referenced_table_names = [referenced_table_base_name, "#{referenced_table_base_name}s"] + normalized_table_name = model_declaration.source_table_name.split('.').last + return false unless normalized_table_name + + list_of_referenced_table_names.includes?(normalized_table_name.downcase) end private def project_relative_path(file_path : String) : String diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations.cr new file mode 100644 index 0000000..eb2acb4 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations.cr @@ -0,0 +1,157 @@ +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks::GrantTenancy + class CollectProjectGrantTenancyDeclarations < Crystal::Visitor + getter list_of_model_declarations : Array(GrantTenantModelDeclaration) + getter? uses_row_tenancy : Bool + getter? uses_schema_tenancy : Bool + + @list_of_model_stack = [] of GrantTenantModelDeclaration + @list_of_namespace_segments = [] of String + @list_of_namespace_stack_sizes = [] of Int32 + @method_depth = 0 + @macro_depth = 0 + @next_class_table_name : String? + + def initialize + @list_of_model_declarations = [] of GrantTenantModelDeclaration + @uses_row_tenancy = false + @uses_schema_tenancy = false + end + + def self.for_source(content : String) : CollectProjectGrantTenancyDeclarations + collector = new + collector.accept(Crystal::Parser.new(content).parse) + collector + rescue Crystal::SyntaxException + new + end + + def visit(node : Crystal::ASTNode) : Bool + true + end + + def visit(node : Crystal::ModuleDef) : Bool + @list_of_namespace_stack_sizes << @list_of_namespace_segments.size + @list_of_namespace_segments.concat(node.name.names) + true + end + + def end_visit(node : Crystal::ModuleDef) : Nil + restore_namespace_stack + end + + def visit(node : Crystal::ClassDef) : Bool + class_name = node.name.names.last + qualified_model_name = if node.name.global? + node.name.names.join("::") + else + (@list_of_namespace_segments + node.name.names).join("::") + end + superclass_reference_name = SourceNode.model_name_for_receiver(node.superclass) + model_declaration = GrantTenantModelDeclaration.new( + qualified_model_name, + class_name, + superclass_reference_name, + ) + if table_name = @next_class_table_name + model_declaration.set_source_table_name(table_name) + end + @next_class_table_name = nil + @list_of_model_stack << model_declaration + true + end + + def end_visit(node : Crystal::ClassDef) : Nil + model_declaration = @list_of_model_stack.pop? + @list_of_model_declarations << model_declaration if model_declaration + end + + def visit(node : Crystal::Def) : Bool + @method_depth += 1 + true + end + + def end_visit(node : Crystal::Def) : Nil + @method_depth -= 1 + end + + def visit(node : Crystal::Macro) : Bool + @macro_depth += 1 + true + end + + def end_visit(node : Crystal::Macro) : Nil + @macro_depth -= 1 + end + + def visit(node : Crystal::Annotation) : Bool + if node.path.names == ["Grant", "Table"] + annotation_name = node.named_args.try(&.find { |argument| argument.name == "name" }) + @next_class_table_name = SourceNode.literal_name(annotation_name.try(&.value)) + end + true + end + + def visit(node : Crystal::Call) : Bool + if SourceNode.full_call_name(node) == "Grant::SchemaTenant.with" + @uses_schema_tenancy = true + end + + return true unless current_class_body? + + model_declaration = @list_of_model_stack.last + case node.name + when "multitenant" + return true unless node.obj.nil? + + model_declaration.capture_multitenant_column(SourceNode.literal_name(node.args.first?)) + @uses_row_tenancy = true + when "schema_tenant_excluded" + return true unless node.obj.nil? + + model_declaration.mark_schema_tenant_excluded + @uses_schema_tenancy = true + when "column" + return true unless node.obj.nil? + + column_declaration = column_declaration_from(node.args.first?) + model_declaration.add_column_declaration(*column_declaration) if column_declaration + when "table" + return true unless node.obj.nil? + + table_name = SourceNode.literal_name(node.args.first?) + model_declaration.set_source_table_name(table_name) if table_name + end + + true + end + + private def current_class_body? : Bool + !@list_of_model_stack.empty? && @method_depth == 0 && @macro_depth == 0 + end + + private def column_declaration_from(node : Crystal::ASTNode?) : Tuple(String, Crystal::ASTNode)? + case node + when Crystal::TypeDeclaration + variable = node.var + return {variable.name, variable} if variable.is_a?(Crystal::Var) + when Crystal::SymbolLiteral + return {node.value, node} + when Crystal::Var + return {node.name, node} + when Crystal::Path + return {node.names.last, node} unless node.names.empty? + end + + nil + end + + private def restore_namespace_stack : Nil + previous_size = @list_of_namespace_stack_sizes.pop? + return unless previous_size + + @list_of_namespace_segments = @list_of_namespace_segments[0, previous_size] + end + end +end diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/grant_tenant_model_declaration.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/grant_tenant_model_declaration.cr new file mode 100644 index 0000000..4ae14ce --- /dev/null +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/grant_tenant_model_declaration.cr @@ -0,0 +1,50 @@ +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks::GrantTenancy + class GrantTenantModelDeclaration + getter qualified_model_name : String + getter class_name : String + getter superclass_reference_name : String? + getter source_table_name : String + getter tenant_column_name : String? + getter list_of_column_declarations : Array(Tuple(String, Crystal::ASTNode)) + getter? has_multitenant_macro : Bool + getter? schema_tenant_excluded : Bool + + def initialize( + @qualified_model_name : String, + @class_name : String, + @superclass_reference_name : String?, + ) + @source_table_name = SourceNode.default_table_name(@class_name) + @tenant_column_name = nil + @list_of_column_declarations = [] of Tuple(String, Crystal::ASTNode) + @has_multitenant_macro = false + @schema_tenant_excluded = false + end + + def capture_multitenant_column(column_name : String?) : Nil + @has_multitenant_macro = true + @tenant_column_name = column_name + end + + def mark_schema_tenant_excluded : Nil + @schema_tenant_excluded = true + end + + def set_source_table_name(table_name : String) : Nil + @source_table_name = table_name + end + + def add_column_declaration(column_name : String, source_node : Crystal::ASTNode) : Nil + @list_of_column_declarations << {column_name, source_node} + end + + def grant_model? : Bool + has_multitenant_macro? || schema_tenant_excluded? || + !@list_of_column_declarations.empty? || + @superclass_reference_name == "Grant::Base" || + @source_table_name != SourceNode.default_table_name(@class_name) + end + end +end diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/source_node.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/source_node.cr new file mode 100644 index 0000000..26a0775 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/source_node.cr @@ -0,0 +1,79 @@ +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks::GrantTenancy::SourceNode + def self.full_call_name(call : Crystal::Call) : String + receiver_name = receiver_path_name(call.obj) + return call.name unless receiver_name + + "#{receiver_name}.#{call.name}" + end + + def self.model_name_for_receiver(node : Crystal::ASTNode?) : String? + case node + when Crystal::Path + node.names.join("::") + when Crystal::Call + model_name_for_receiver(node.obj) + else + nil + end + end + + def self.literal_name(node : Crystal::ASTNode?) : String? + case node + when Crystal::SymbolLiteral + node.value + when Crystal::StringLiteral + node.value + when Crystal::Path + node.names.last? + when Crystal::Var + node.name + else + nil + end + end + + def self.default_table_name(class_name : String) : String + characters = class_name.chars + + String.build do |table_name| + characters.each_with_index do |character, character_index| + previous_character = character_index > 0 ? characters[character_index - 1] : nil + next_character = characters[character_index + 1]? + starts_new_word = character_is_uppercase?(character) && previous_character && + (character_is_lowercase_or_digit?(previous_character) || + (character_is_uppercase?(previous_character) && next_character && character_is_lowercase?(next_character))) + + table_name << '_' if starts_new_word && !table_name.empty? + table_name << character.downcase + end + end + end + + private def self.receiver_path_name(node : Crystal::ASTNode?) : String? + case node + when Crystal::Path + node.names.join("::") + when Crystal::Call + receiver_name = receiver_path_name(node.obj) + return node.name unless receiver_name + + "#{receiver_name}.#{node.name}" + else + nil + end + end + + private def self.character_is_uppercase?(character : Char) : Bool + character >= 'A' && character <= 'Z' + end + + private def self.character_is_lowercase?(character : Char) : Bool + character >= 'a' && character <= 'z' + end + + private def self.character_is_lowercase_or_digit?(character : Char) : Bool + character_is_lowercase?(character) || (character >= '0' && character <= '9') + end +end diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls.cr new file mode 100644 index 0000000..d517072 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls.cr @@ -0,0 +1,73 @@ +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks::GrantTenancy + class VisitChainableUnscopedModelCalls < Crystal::Visitor + class Occurrence + getter call : Crystal::Call + getter model_reference_name : String + getter? has_bulk_write_after : Bool + + def initialize(@call : Crystal::Call, @model_reference_name : String, @has_bulk_write_after : Bool) + end + end + + getter list_of_chainable_unscoped_calls : Array(Occurrence) + getter list_of_block_unscoped_calls : Array(Crystal::Call) + + @list_of_call_stack = [] of Crystal::Call + + def initialize(@project_state : AmberLSP::LibraryRulePacks::DetermineProjectRulePackState) + @list_of_chainable_unscoped_calls = [] of Occurrence + @list_of_block_unscoped_calls = [] of Crystal::Call + end + + def visit(node : Crystal::ASTNode) : Bool + true + end + + def visit(node : Crystal::Call) : Bool + if node.name == "unscoped" + if model_reference_name = SourceNode.model_name_for_receiver(node.obj) + if @project_state.has_row_tenant_model?(model_reference_name) + if node.block + @list_of_block_unscoped_calls << node + else + @list_of_chainable_unscoped_calls << Occurrence.new( + node, + model_reference_name, + has_bulk_write_after?(node, model_reference_name), + ) + end + end + end + end + + @list_of_call_stack << node + true + end + + def end_visit(node : Crystal::Call) : Nil + @list_of_call_stack.pop + end + + private def has_bulk_write_after?(unscoped_call : Crystal::Call, model_reference_name : String) : Bool + @list_of_call_stack.any? do |parent_call| + next false unless {"update_all", "delete_all"}.includes?(parent_call.name) + + parent_model_reference_name = SourceNode.model_name_for_receiver(parent_call.obj) + next false unless parent_model_reference_name == model_reference_name + + receiver_chain_has_chainable_unscoped_call?(parent_call.obj) + end + end + + private def receiver_chain_has_chainable_unscoped_call?(node : Crystal::ASTNode?) : Bool + case node + when Crystal::Call + (node.name == "unscoped" && node.block.nil?) || receiver_chain_has_chainable_unscoped_call?(node.obj) + else + false + end + end + end +end diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_schema_queries_outside_tenant_blocks.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_schema_queries_outside_tenant_blocks.cr new file mode 100644 index 0000000..ca01952 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_schema_queries_outside_tenant_blocks.cr @@ -0,0 +1,77 @@ +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks::GrantTenancy + class VisitGrantSchemaQueriesOutsideTenantBlocks < Crystal::Visitor + LIST_OF_QUERY_METHOD_NAMES = { + "all", "where", "find", "find!", "find_by", "find_by!", "first", "first?", + "last", "last?", "count", "exists?", "pluck", "sum", "average", "minimum", "maximum", + } + + getter list_of_schema_queries_outside_tenant_blocks : Array(Crystal::Call) + + @schema_tenant_block_depth = 0 + @list_of_call_stack = [] of Crystal::Call + @list_of_block_scope_changes = [] of Bool + @list_of_method_scope_snapshots = [] of Int32 + + def initialize(@project_state : AmberLSP::LibraryRulePacks::DetermineProjectRulePackState) + @list_of_schema_queries_outside_tenant_blocks = [] of Crystal::Call + end + + def visit(node : Crystal::ASTNode) : Bool + true + end + + def visit(node : Crystal::Def) : Bool + @list_of_method_scope_snapshots << @schema_tenant_block_depth + @schema_tenant_block_depth = 0 + true + end + + def end_visit(node : Crystal::Def) : Nil + previous_scope_depth = @list_of_method_scope_snapshots.pop? + @schema_tenant_block_depth = previous_scope_depth if previous_scope_depth + end + + def visit(node : Crystal::Block) : Bool + enters_schema_tenant_block = node.call.try do |block_call| + SourceNode.full_call_name(block_call) == "Grant::SchemaTenant.with" + end || false + @schema_tenant_block_depth += 1 if enters_schema_tenant_block + @list_of_block_scope_changes << enters_schema_tenant_block + true + end + + def end_visit(node : Crystal::Block) : Nil + enters_schema_tenant_block = @list_of_block_scope_changes.pop? + @schema_tenant_block_depth -= 1 if enters_schema_tenant_block + end + + def visit(node : Crystal::Call) : Bool + model_reference_name = SourceNode.model_name_for_receiver(node.obj) + if model_reference_name && + LIST_OF_QUERY_METHOD_NAMES.includes?(node.name) && + @schema_tenant_block_depth == 0 && + @project_state.has_non_excluded_schema_model?(model_reference_name) && + !nested_inside_schema_model_query?(model_reference_name) + @list_of_schema_queries_outside_tenant_blocks << node + end + + @list_of_call_stack << node + true + end + + def end_visit(node : Crystal::Call) : Nil + @list_of_call_stack.pop + end + + private def nested_inside_schema_model_query?(model_reference_name : String) : Bool + @list_of_call_stack.any? do |parent_call| + next false unless LIST_OF_QUERY_METHOD_NAMES.includes?(parent_call.name) + + parent_model_reference_name = SourceNode.model_name_for_receiver(parent_call.obj) + parent_model_reference_name == model_reference_name + end + end + end +end diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_tenant_clear_calls.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_tenant_clear_calls.cr new file mode 100644 index 0000000..b6b813f --- /dev/null +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_tenant_clear_calls.cr @@ -0,0 +1,22 @@ +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks::GrantTenancy + class VisitGrantTenantClearCalls < Crystal::Visitor + getter list_of_tenant_clear_calls : Array(Crystal::Call) + + def initialize + @list_of_tenant_clear_calls = [] of Crystal::Call + end + + def visit(node : Crystal::ASTNode) : Bool + true + end + + def visit(node : Crystal::Call) : Bool + if SourceNode.full_call_name(node) == "Grant::Tenant.clear" + @list_of_tenant_clear_calls << node + end + true + end + end +end diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites.cr new file mode 100644 index 0000000..e4548b4 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites.cr @@ -0,0 +1,123 @@ +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks::GrantTenancy + class VisitRawConnectionSqlCallSites < Crystal::Visitor + class Occurrence + getter call : Crystal::Call + getter sql_literal : Crystal::StringLiteral + + def initialize(@call : Crystal::Call, @sql_literal : Crystal::StringLiteral) + end + end + + getter list_of_raw_connection_sql_call_sites : Array(Occurrence) + + @list_of_active_database_handle_names = [] of String + @list_of_block_handle_scope_sizes = [] of Int32 + + def initialize(@project_state : AmberLSP::LibraryRulePacks::DetermineProjectRulePackState) + @list_of_raw_connection_sql_call_sites = [] of Occurrence + end + + def visit(node : Crystal::ASTNode) : Bool + true + end + + def visit(node : Crystal::Block) : Bool + list_of_new_handle_names = [] of String + if adapter_open_call?(node.call) + node.args.each { |argument| list_of_new_handle_names << argument.name } + end + + @list_of_active_database_handle_names.concat(list_of_new_handle_names) + @list_of_block_handle_scope_sizes << list_of_new_handle_names.size + true + end + + def end_visit(node : Crystal::Block) : Nil + added_handle_count = @list_of_block_handle_scope_sizes.pop? + return unless added_handle_count && added_handle_count > 0 + + @list_of_active_database_handle_names = @list_of_active_database_handle_names[0, + @list_of_active_database_handle_names.size - added_handle_count] + end + + def visit(node : Crystal::Call) : Bool + return true unless raw_connection_sql_call?(node) + + sql_literal = node.args.first?.as?(Crystal::StringLiteral) + return true unless sql_literal + return true unless sql_literal_names_tenant_table?(sql_literal.value) + + @list_of_raw_connection_sql_call_sites << Occurrence.new(node, sql_literal) + true + end + + private def raw_connection_sql_call?(call : Crystal::Call) : Bool + if {"exec", "query", "scalar"}.includes?(call.name) + return database_handle_call?(call.obj) + end + + if {"execute", "exec_query", "select_all", "select_one", "select_value", + "select_values", "select_rows", "with_result_set"}.includes?(call.name) + return grant_connection_facade_call?(call) + end + + false + end + + private def database_handle_call?(node : Crystal::ASTNode?) : Bool + if node.is_a?(Crystal::Var) + return @list_of_active_database_handle_names.includes?(node.name) + end + + false + end + + private def grant_connection_facade_call?(call : Crystal::Call) : Bool + receiver_call = call.obj.as?(Crystal::Call) + return false unless receiver_call + + receiver_call.name == "connection" && + SourceNode.full_call_name(receiver_call) == "Grant.connection" + end + + private def adapter_open_call?(call : Crystal::Call?) : Bool + return false unless call + return false unless call.name == "open" + + receiver_has_adapter_call?(call.obj) || receiver_has_connection_registry_path?(call.obj) + end + + private def receiver_has_adapter_call?(node : Crystal::ASTNode?) : Bool + case node + when Crystal::Call + node.name == "adapter" || receiver_has_adapter_call?(node.obj) + when Crystal::Var + node.name == "adapter" + else + false + end + end + + private def receiver_has_connection_registry_path?(node : Crystal::ASTNode?) : Bool + case node + when Crystal::Path + node.names == ["Grant", "Connections"] + when Crystal::Call + receiver_has_connection_registry_path?(node.obj) || + node.args.any? { |argument| receiver_has_connection_registry_path?(argument) } + else + false + end + end + + private def sql_literal_names_tenant_table?(sql_literal : String) : Bool + list_of_sql_tokens = sql_literal.downcase.split(/[^a-z0-9_]+/) + @project_state.list_of_row_tenant_table_names.any? do |table_name| + normalized_table_name = table_name.split('.').last + normalized_table_name && list_of_sql_tokens.includes?(normalized_table_name.downcase) + end + end + end +end diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_spawn_calls_inside_grant_tenant_blocks.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_spawn_calls_inside_grant_tenant_blocks.cr new file mode 100644 index 0000000..1d942f2 --- /dev/null +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_spawn_calls_inside_grant_tenant_blocks.cr @@ -0,0 +1,62 @@ +require "compiler/crystal/syntax" + +module AmberLSP::LibraryRulePacks::GrantTenancy + class VisitSpawnCallsInsideGrantTenantBlocks < Crystal::Visitor + getter list_of_spawn_calls_inside_tenant_blocks : Array(Crystal::Call) + + @tenant_block_depth = 0 + @list_of_block_scope_changes = [] of Bool + @list_of_method_scope_snapshots = [] of Int32 + @macro_depth = 0 + + def initialize + @list_of_spawn_calls_inside_tenant_blocks = [] of Crystal::Call + end + + def visit(node : Crystal::ASTNode) : Bool + true + end + + def visit(node : Crystal::Def) : Bool + @list_of_method_scope_snapshots << @tenant_block_depth + @tenant_block_depth = 0 + true + end + + def end_visit(node : Crystal::Def) : Nil + previous_scope_depth = @list_of_method_scope_snapshots.pop? + @tenant_block_depth = previous_scope_depth if previous_scope_depth + end + + def visit(node : Crystal::Macro) : Bool + @macro_depth += 1 + true + end + + def end_visit(node : Crystal::Macro) : Nil + @macro_depth -= 1 + end + + def visit(node : Crystal::Block) : Bool + call_is_tenant_scope = node.call.try do |block_call| + {"Grant::Tenant.with", "Grant::SchemaTenant.with"}.includes?(SourceNode.full_call_name(block_call)) + end || false + enters_tenant_scope = call_is_tenant_scope && @macro_depth == 0 + @tenant_block_depth += 1 if enters_tenant_scope + @list_of_block_scope_changes << enters_tenant_scope + true + end + + def end_visit(node : Crystal::Block) : Nil + scope_change = @list_of_block_scope_changes.pop? + @tenant_block_depth -= 1 if scope_change + end + + def visit(node : Crystal::Call) : Bool + if node.name == "spawn" && @tenant_block_depth > 0 && @macro_depth == 0 + @list_of_spawn_calls_inside_tenant_blocks << node + end + true + end + end +end diff --git a/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr b/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr index 1becfd1..3da1612 100644 --- a/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr +++ b/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr @@ -1,3 +1,5 @@ +require "log" + module AmberLSP::LibraryRulePacks class LoadRulePacksForProject def initialize(@project_context : AmberLSP::ProjectContext) @@ -24,32 +26,28 @@ module AmberLSP::LibraryRulePacks private def dependency_pack_paths : Array(String) list_of_library_paths = Dir.glob(File.join(@project_context.root_path, "lib", "*")).sort list_of_library_paths.flat_map do |library_path| - Dir.glob(File.join(library_path, ".claude", "rules", "*.yml")) + Dir.glob(File.join(library_path, ".amber-lsp", "packs", "*.yml")) end.sort end private def project_pack_paths : Array(String) - Dir.glob(File.join(@project_context.root_path, ".claude", "rules", "*.yml")).sort + Dir.glob(File.join(@project_context.root_path, ".amber-lsp", "packs", "*.yml")).sort end private def load_rule_pack(pack_path : String) : DescribeLibraryRulePack? rule_pack = DescribeLibraryRulePack.from_yaml(File.read(pack_path)) return rule_pack if rule_pack_is_valid?(rule_pack) - STDERR.puts "WARNING: Ignoring invalid amber-lsp rule pack at #{pack_path}." + Log.warn { "Ignoring invalid amber-lsp rule pack at #{pack_path}." } nil rescue ex - STDERR.puts "WARNING: Could not load amber-lsp rule pack at #{pack_path}: #{ex.message}" + Log.warn { "Could not load amber-lsp rule pack at #{pack_path}: #{ex.message}" } nil end private def rule_pack_is_valid?(rule_pack : DescribeLibraryRulePack) : Bool return false unless rule_pack.is_valid? - rule_pack.modes_by_name.each_value do |mode| - mode.list_of_evidence_patterns.each { |pattern| Regex.new(pattern) } - end - rule_pack.list_of_rules.all? do |rule| next false unless {"error", "warning", "info", "hint"}.includes?(rule.severity_name.downcase) @@ -59,23 +57,26 @@ module AmberLSP::LibraryRulePacks true when "file_requires" !rule.check.required_regex_pattern.empty? && - (rule.check.trigger_regex_pattern.empty? || begin + !rule.check.trigger_regex_pattern.empty? && + begin Regex.new(rule.check.trigger_regex_pattern) + Regex.new(rule.check.required_regex_pattern) true - end) && begin - Regex.new(rule.check.required_regex_pattern) - true - end + end when "call_outside_block" !rule.check.list_of_source_globs.empty? && !rule.check.list_of_query_method_names.empty? && (!rule.check.required_block_call_name.empty? || !rule.check.escape_block_call_name.empty?) + when "crystal_ast" + {"chained_unscoped_in_request_code", "chained_unscoped_bulk_write", + "chained_unscoped_on_tenant_model", "unscoped_block_in_request_code", + "spawn_inside_tenant_block", "tenant_column_without_multitenant", + "raw_connection_sql_on_tenant_table", "tenant_clear_in_app_code", + "schema_query_outside_tenant"}.includes?(rule.check.operation_name) when "project_conflict" case rule.check.project_condition when "mixed_modes" rule.list_of_mode_names.size > 1 - when "evidence_without_declaration" - rule.list_of_mode_names.size == 1 else false end diff --git a/src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts.cr b/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr similarity index 68% rename from src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts.cr rename to src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr index 1d17509..88bd929 100644 --- a/src/amber_lsp/library_rule_packs/print_declared_rule_pack_contexts.cr +++ b/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr @@ -1,5 +1,5 @@ module AmberLSP::LibraryRulePacks - class PrintDeclaredRulePackContexts + class PrintDetectedRulePackContexts def initialize(@list_of_arguments : Array(String)) end @@ -14,8 +14,7 @@ module AmberLSP::LibraryRulePacks project_state = DetermineProjectRulePackState.new(project_context, rule_pack, "", "") next unless project_state.has_any_applicable_mode? - append_declared_mode_contexts(list_of_output_lines, rule_pack, project_state) - append_undeclared_feature_warning(list_of_output_lines, rule_pack, project_state) + append_detected_mode_contexts(list_of_output_lines, rule_pack, project_state) end STDOUT.puts(list_of_output_lines.join('\n')) unless list_of_output_lines.empty? @@ -57,34 +56,18 @@ module AmberLSP::LibraryRulePacks end end - private def append_declared_mode_contexts( + private def append_detected_mode_contexts( list_of_output_lines : Array(String), rule_pack : DescribeLibraryRulePack, project_state : DetermineProjectRulePackState, ) : Nil rule_pack.modes_by_name.each do |mode_name, mode| - next unless project_state.is_mode_declared?(mode_name) + next unless project_state.mode_detected?(mode_name) next if mode.guidance_text.strip.empty? list_of_output_lines << "#{rule_pack.pack_id} (#{mode_name})" list_of_output_lines.concat(mode.guidance_text.lines.map(&.rstrip)) end end - - private def append_undeclared_feature_warning( - list_of_output_lines : Array(String), - rule_pack : DescribeLibraryRulePack, - project_state : DetermineProjectRulePackState, - ) : Nil - list_of_undeclared_mode_names = rule_pack.modes_by_name.keys.select do |mode_name| - project_state.has_evidence_for_mode?(mode_name) && !project_state.is_mode_declared?(mode_name) - end - return if list_of_undeclared_mode_names.empty? - - list_of_key_paths = list_of_undeclared_mode_names.compact_map do |mode_name| - rule_pack.modes_by_name[mode_name]?.try(&.declaration.key_path) - end - list_of_output_lines << "warning: #{rule_pack.pack_id} feature is used but shard.yml does not declare #{list_of_key_paths.join(", ")}." - end end end diff --git a/src/amber_lsp/project_context.cr b/src/amber_lsp/project_context.cr index 3064f9e..4b550ee 100644 --- a/src/amber_lsp/project_context.cr +++ b/src/amber_lsp/project_context.cr @@ -6,13 +6,10 @@ module AmberLSP getter? amber_project : Bool getter shard_name : String? - @shard_configuration : YAML::Any? - def initialize( @root_path : String, @amber_project : Bool = false, @shard_name : String? = nil, - @shard_configuration : YAML::Any? = nil, ) end @@ -31,28 +28,11 @@ module AmberLSP root_path, amber_project: is_amber, shard_name: shard_name, - shard_configuration: shard_configuration, ) rescue YAML::ParseException ProjectContext.new(root_path, amber_project: false) end - def has_shard_declaration?(key_path : String, expected_value : String) : Bool - shard_configuration = @shard_configuration - return false unless shard_configuration - - current_value = shard_configuration - key_path.split('.').each do |key| - next_value = current_value[key]? - return false unless next_value - current_value = next_value - end - - current_value.as_s? == expected_value - rescue TypeCastError - false - end - private def self.has_amber_dependency?(shard_configuration : YAML::Any) : Bool dependencies = shard_configuration["dependencies"]? return false unless dependencies From 95f04cfb837db63aca338e7790fc01b8ffad991f Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Wed, 23 Sep 2026 21:07:53 -0400 Subject: [PATCH 03/69] fix(amber-lsp): narrow rule pack rescues --- spec/amber_lsp/rule_packs_spec.cr | 51 +++++++++++++++++++ .../visit_chainable_unscoped_model_calls.cr | 2 +- .../visit_raw_connection_sql_call_sites.cr | 2 +- .../load_rule_packs_for_project.cr | 12 +++-- .../print_detected_rule_pack_contexts.cr | 11 +++- 5 files changed, 71 insertions(+), 7 deletions(-) diff --git a/spec/amber_lsp/rule_packs_spec.cr b/spec/amber_lsp/rule_packs_spec.cr index 51ebe49..6d4244e 100644 --- a/spec/amber_lsp/rule_packs_spec.cr +++ b/spec/amber_lsp/rule_packs_spec.cr @@ -64,6 +64,15 @@ def tenancy_diagnostic_codes(diagnostics : Array(AmberLSP::Rules::Diagnostic)) : diagnostics.map(&.code) end +class UnexpectedRulePackReadError < Exception +end + +class RaiseUnexpectedRulePackReadError < AmberLSP::LibraryRulePacks::LoadRulePacksForProject + protected def read_rule_pack_contents(pack_path : String) : String + raise UnexpectedRulePackReadError.new("unexpected read failure") + end +end + describe "AmberLSP Grant tenancy rule pack v2" do before_each do AmberLSP::Rules::RuleRegistry.clear @@ -82,6 +91,48 @@ describe "AmberLSP Grant tenancy rule pack v2" do end end + it "skips malformed and unreadable pack files" do + with_tempdir do |root| + pack_directory = File.join(root, "lib", "grant", ".amber-lsp", "packs") + Dir.mkdir_p(pack_directory) + File.write(File.join(pack_directory, "malformed.yml"), "pack: [\n") + Dir.mkdir(File.join(pack_directory, "unreadable.yml")) + + project_context = AmberLSP::ProjectContext.new(root) + list_of_rule_packs = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs + + list_of_rule_packs.should be_empty + end + end + + it "propagates unexpected pack read errors" do + with_tempdir do |root| + pack_path = File.join(root, "lib", "grant", ".amber-lsp", "packs", "tenancy.yml") + Dir.mkdir_p(File.dirname(pack_path)) + File.write(pack_path, "pack: test\n") + + project_context = AmberLSP::ProjectContext.new(root) + + expect_raises(UnexpectedRulePackReadError, "unexpected read failure") do + RaiseUnexpectedRulePackReadError.new(project_context).load_rule_packs + end + end + end + + it "logs the exception class when the context command rejects its arguments" do + log_backend = Log::MemoryBackend.new + Log.setup(:error, log_backend) + + begin + result = AmberLSP::LibraryRulePacks::PrintDetectedRulePackContexts.new(["--unexpected"]).perform + + result.should eq(1) + log_backend.entries.last.message.should contain("ArgumentError") + ensure + Log.setup + end + end + it "gives agents the Grant runtime context for detected modes" do with_tempdir do |root| install_tenancy_fixture_app(root, "row_app") diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls.cr index d517072..26b4dcc 100644 --- a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls.cr +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_chainable_unscoped_model_calls.cr @@ -2,7 +2,7 @@ require "compiler/crystal/syntax" module AmberLSP::LibraryRulePacks::GrantTenancy class VisitChainableUnscopedModelCalls < Crystal::Visitor - class Occurrence + struct Occurrence getter call : Crystal::Call getter model_reference_name : String getter? has_bulk_write_after : Bool diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites.cr index e4548b4..2a07c9d 100644 --- a/src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites.cr +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/visit_raw_connection_sql_call_sites.cr @@ -2,7 +2,7 @@ require "compiler/crystal/syntax" module AmberLSP::LibraryRulePacks::GrantTenancy class VisitRawConnectionSqlCallSites < Crystal::Visitor - class Occurrence + struct Occurrence getter call : Crystal::Call getter sql_literal : Crystal::StringLiteral diff --git a/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr b/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr index 3da1612..5fb79f2 100644 --- a/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr +++ b/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr @@ -35,16 +35,22 @@ module AmberLSP::LibraryRulePacks end private def load_rule_pack(pack_path : String) : DescribeLibraryRulePack? - rule_pack = DescribeLibraryRulePack.from_yaml(File.read(pack_path)) + rule_pack = DescribeLibraryRulePack.from_yaml(read_rule_pack_contents(pack_path)) return rule_pack if rule_pack_is_valid?(rule_pack) Log.warn { "Ignoring invalid amber-lsp rule pack at #{pack_path}." } nil - rescue ex - Log.warn { "Could not load amber-lsp rule pack at #{pack_path}: #{ex.message}" } + rescue ex : YAML::ParseException | IO::Error + Log.warn(exception: ex) do + "Could not load amber-lsp rule pack at #{pack_path} (#{ex.class}): #{ex.message}" + end nil end + protected def read_rule_pack_contents(pack_path : String) : String + File.read(pack_path) + end + private def rule_pack_is_valid?(rule_pack : DescribeLibraryRulePack) : Bool return false unless rule_pack.is_valid? diff --git a/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr b/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr index 88bd929..f64c0bf 100644 --- a/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr +++ b/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr @@ -1,5 +1,10 @@ +require "log" +require "yaml" + module AmberLSP::LibraryRulePacks class PrintDetectedRulePackContexts + Log = ::Log.for(self) + def initialize(@list_of_arguments : Array(String)) end @@ -19,8 +24,10 @@ module AmberLSP::LibraryRulePacks STDOUT.puts(list_of_output_lines.join('\n')) unless list_of_output_lines.empty? 0 - rescue ex - STDERR.puts "amber-lsp context failed: #{ex.message}" + rescue ex : ArgumentError | YAML::ParseException | IO::Error + Log.error(exception: ex) do + "amber-lsp context failed: #{ex.class}: #{ex.message}" + end 1 end From 83344e7e8c3f3246f92410d278a739db79177717 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Sat, 3 Oct 2026 11:57:11 -0400 Subject: [PATCH 04/69] feat: add Amber agent setup loop --- README.md | 13 ++ .../check_file_for_diagnostics_spec.cr | 34 ++++ spec/commands/setup_agent_spec.cr | 175 ++++++++++++++++++ spec/commands/setup_lsp_spec.cr | 17 ++ src/amber_cli.cr | 2 + .../agent/resolve_compiler_for_agent_loop.cr | 24 +++ src/amber_cli/commands/new.cr | 1 + src/amber_cli/commands/setup_agent.cr | 165 +++++++++++++++++ src/amber_cli/commands/setup_lsp.cr | 19 +- .../templates/agent/amber-agent-hook | 144 ++++++++++++++ src/amber_lsp.cr | 12 +- src/amber_lsp/check_file_for_diagnostics.cr | 26 +++ 12 files changed, 624 insertions(+), 8 deletions(-) create mode 100644 spec/amber_lsp/check_file_for_diagnostics_spec.cr create mode 100644 spec/commands/setup_agent_spec.cr create mode 100644 spec/commands/setup_lsp_spec.cr create mode 100644 src/amber_cli/agent/resolve_compiler_for_agent_loop.cr create mode 100644 src/amber_cli/commands/setup_agent.cr create mode 100644 src/amber_cli/templates/agent/amber-agent-hook create mode 100644 src/amber_lsp/check_file_for_diagnostics.cr diff --git a/README.md b/README.md index 1985bec..a674c61 100644 --- a/README.md +++ b/README.md @@ -143,6 +143,7 @@ SQL migration to `db/migrations/`, and the resource route to `config/routes.cr`. | `amber assets check` | Supported | Verify manifest, bytes, integrity, MIME, and compressed output without changing it | | `amber new APP --type native` | Preview | Not part of the beta platform guarantee | | `amber setup:lsp` | Available | Configure the bundled diagnostics LSP | +| `amber setup:agent` (`amber agent`) | Available | Install optional Claude Code and Codex feedback hooks | Run `amber --help` or `amber COMMAND --help` for command syntax. The detailed [web-app walkthrough](docs/BETA_WEB_APP.md) and @@ -177,6 +178,18 @@ amber setup:lsp See the [LSP setup guide](https://github.com/amberframework/amber/blob/v2.0.0-beta.5/docs/guides/lsp-setup.md). +## AI agent setup + +From an Amber V2 project, run `amber setup:agent` to install Claude Code and +Codex hooks, the project-local `bin/amber-agent-hook`, agent instructions, and +Amber LSP discovery. Existing hook settings and instructions are merged; the +command is safe to run again. The post-edit hook formats and checks only the +changed Crystal file with `amber-lsp --check`; the stop hook asks +`crystal-alpha watch build` for compiler feedback, or checks the target in +`shard.yml` with `--no-codegen` when no watcher is running. Install +`crystal-alpha` and `amber-lsp` for the full loop. Codex may ask you to trust +new project hooks before it runs them. + ## Contributing ```bash diff --git a/spec/amber_lsp/check_file_for_diagnostics_spec.cr b/spec/amber_lsp/check_file_for_diagnostics_spec.cr new file mode 100644 index 0000000..220cdc7 --- /dev/null +++ b/spec/amber_lsp/check_file_for_diagnostics_spec.cr @@ -0,0 +1,34 @@ +require "./spec_helper" +require "../../src/amber_lsp/rules/controllers/naming_rule" +require "../../src/amber_lsp/check_file_for_diagnostics" + +describe AmberLSP::CheckFileForDiagnostics do + it "prints a file and line for a real rule violation" do + with_tempdir do |project| + Dir.mkdir_p(File.join(project, "src/controllers")) + file_path = File.join(project, "src/controllers/users_controller.cr") + File.write(file_path, "class UsersHandler < Amber::Controller::Base\nend\n") + output = IO::Memory.new + errors = IO::Memory.new + previous_directory = Dir.current + begin + Dir.cd(project) + status = AmberLSP::CheckFileForDiagnostics.new(file_path, output, errors).perform + status.should eq(1) + output.to_s.should contain("#{file_path}:1: error: amber/controller-naming") + errors.to_s.should be_empty + ensure + Dir.cd(previous_directory) + end + end + end + + it "reports a missing file as a check error" do + with_tempdir do |project| + errors = IO::Memory.new + status = AmberLSP::CheckFileForDiagnostics.new(File.join(project, "missing.cr"), IO::Memory.new, errors).perform + status.should eq(2) + errors.to_s.should contain("file not found") + end + end +end diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr new file mode 100644 index 0000000..6e5470b --- /dev/null +++ b/spec/commands/setup_agent_spec.cr @@ -0,0 +1,175 @@ +require "../amber_cli_spec" +require "../../src/amber_cli/commands/setup_agent" + +describe "amber setup:agent" do + it "registers the setup command and short alias" do + AmberCLI::Core::CommandRegistry.find_command("setup:agent").should_not be_nil + AmberCLI::Core::CommandRegistry.find_command("agent").should eq(AmberCLI::Core::CommandRegistry.find_command("setup:agent")) + end + + it "resolves the alpha override, aliases, and stock compiler in order" do + calls = [] of String + lookup = ->(command : String) do + calls << command + {"acrystal" => "/tools/acrystal", "crystal" => "/tools/crystal"}[command]? + end + + override = AmberCLI::Agent::ResolveCompilerForAgentLoop.new("/custom/alpha", lookup) + override.perform.should eq("/custom/alpha") + calls.should be_empty + + alias_choice = AmberCLI::Agent::ResolveCompilerForAgentLoop.new(nil, lookup) + alias_choice.perform.should eq("/tools/acrystal") + alias_choice.stock_compiler?.should be_false + calls.should eq(["crystal-alpha", "acrystal"]) + + stock = AmberCLI::Agent::ResolveCompilerForAgentLoop.new(nil, ->(command : String) { command == "crystal" ? "/tools/crystal" : nil }) + stock.perform.should eq("/tools/crystal") + stock.stock_compiler?.should be_true + end + + it "merges hooks while preserving unrelated settings and remains idempotent" do + existing = %({"permissions":{"allow":["Bash(git status)"]},"hooks":{"PreToolUse":[{"matcher":"Edit|Write|MultiEdit|NotebookEdit","extra":"stay","hooks":[{"type":"command","command":"bin/existing-hook","timeout":42}]}],"SessionStart":[{"hooks":[{"type":"command","command":"bin/start-hook"}]}]}}) + first = AmberCLI::Agent::MergeAgentHooksIntoSettings.new(existing).perform + second = AmberCLI::Agent::MergeAgentHooksIntoSettings.new(first).perform + + second.should eq(first) + first.should contain("Bash(git status)") + first.should contain("bin/existing-hook") + first.should contain("bin/start-hook") + first.should contain(%("extra": "stay")) + first.should contain(%("timeout": 42)) + ["pre", "post", "stop"].each do |event| + first.scan(/bin\/amber-agent-hook #{event}/).size.should eq(1) + end + end + + it "installs the loop into a project without changing existing instructions twice" do + SpecHelper.within_temp_directory do |project| + Dir.mkdir_p("src") + File.write("src/custom_entry.cr", "puts :ok\n") + File.write("shard.yml", "name: my_app\ntargets:\n app_entry:\n main: src/custom_entry.cr\n") + File.write("CLAUDE.md", "# Existing Claude instructions\n") + File.write("AGENTS.md", "# Existing Codex instructions\n") + Dir.mkdir_p(".claude") + Dir.mkdir_p(".codex") + File.write(".claude/settings.json", %({"permissions":{"allow":["Read"]}})) + File.write(".codex/hooks.json", %({"description":"keep this","hooks":{"SessionStart":[]}})) + + command = AmberCLI::Commands::SetupAgentCommand.new("setup:agent") + command.execute + first_claude = File.read(".claude/settings.json") + first_codex = File.read(".codex/hooks.json") + first_instructions = File.read("CLAUDE.md") + first_script = File.read("bin/amber-agent-hook") + command.execute + + File.read(".claude/settings.json").should eq(first_claude) + File.read(".codex/hooks.json").should eq(first_codex) + first_claude.should contain(%("allow": [)) + first_codex.should contain("keep this") + first_codex.should contain(%("SessionStart": [])) + File.read("CLAUDE.md").should eq(first_instructions) + File.read("AGENTS.md").scan(/amber-agent-loop:start/).size.should eq(1) + first_instructions.should contain("# Existing Claude instructions") + first_script.should contain("build --no-codegen 'src/custom_entry.cr'") + File.file?(".lsp.json").should be_true + File.info("bin/amber-agent-hook").permissions.to_i.&(0o111).should_not eq(0) + end + end + + it "runs pre, post, and stop against a fake compiler and LSP" do + SpecHelper.within_temp_directory do |project| + Dir.mkdir_p("src") + File.write("src/my app.cr", "puts :ok\n") + File.write("shard.yml", "name: my_app\ntargets:\n my_app:\n main: src/my_app.cr\n") + File.write("src/my_app.cr", "puts :ok\n") + AmberCLI::Commands::SetupAgentCommand.new("setup:agent").execute + + tools = File.join(project, "fake-tools") + Dir.mkdir_p(tools) + compiler = File.join(tools, "crystal-alpha") + lsp = File.join(tools, "amber-lsp") + log = File.join(project, "calls.log") + File.write(compiler, "#!/bin/sh\nprintf 'compiler %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\nif [ \"$*\" = 'watch build' ]; then\n [ \"${TEST_WATCH_EXIT:-0}\" -eq 0 ] || echo 'watch build failed'\n exit \"${TEST_WATCH_EXIT:-0}\"\nfi\n[ \"${TEST_FALLBACK_EXIT:-0}\" -eq 0 ] || echo 'fallback failed'\nexit \"${TEST_FALLBACK_EXIT:-0}\"\n") + File.write(lsp, "#!/bin/sh\nprintf 'lsp %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\nif [ \"${TEST_LSP_EXIT:-0}\" -ne 0 ]; then\n echo 'lsp violation'\n exit \"$TEST_LSP_EXIT\"\nfi\n") + File.chmod(compiler, 0o755) + File.chmod(lsp, 0o755) + path = "#{tools}:/usr/bin:/bin" + hook = File.join(project, "bin/amber-agent-hook") + + run_hook = ->(event : String, payload : String, watch_exit : String, fallback_exit : String) do + output = IO::Memory.new + errors = IO::Memory.new + status = Process.run(hook, [event], input: IO::Memory.new(payload), output: output, error: errors, + env: {"PATH" => path, "TEST_COMMAND_LOG" => log, "TEST_WATCH_EXIT" => watch_exit, "TEST_FALLBACK_EXIT" => fallback_exit}) + {status.exit_code, output.to_s, errors.to_s} + end + + run_hook.call("pre", "{}", "0", "0")[0].should eq(0) + edited_file = File.join(project, "src/my app.cr") + run_hook.call("post", {"tool_input" => {"file_path" => edited_file}}.to_json, "0", "0")[0].should eq(0) + calls = File.read(log) + calls.should contain("compiler watch hold") + calls.should contain("compiler tool format #{edited_file}") + calls.should contain("lsp --check #{edited_file}") + calls.should_not contain("compiler watch build") + + quoted_file = File.join(project, %(src/odd"file.cr)) + File.write(quoted_file, "puts :ok\n") + run_hook.call("post", {"tool_input" => {"file_path" => quoted_file}}.to_json, "0", "0")[0].should eq(0) + File.read(log).should contain("compiler tool format #{quoted_file}") + + previous_calls = File.read(log) + run_hook.call("post", {"tool_input" => {"file_path" => "README.md"}}.to_json, "0", "0")[0].should eq(0) + File.read(log).should eq(previous_calls) + + another_file = File.join(project, "src/another_file.cr") + File.write(another_file, "puts :ok\n") + patch = "*** Begin Patch\n*** Update File: src/my app.cr\n*** Update File: src/another_file.cr\n*** End Patch" + codex_payload = {"tool_name" => "apply_patch", "tool_input" => {"command" => patch}}.to_json + run_hook.call("post", codex_payload, "0", "0")[0].should eq(0) + codex_calls = File.read(log) + codex_calls.should match(/compiler tool format .*\/src\/another_file\.cr/) + codex_calls.should match(/lsp --check .*\/src\/another_file\.cr/) + + diagnostic_output = IO::Memory.new + diagnostic_errors = IO::Memory.new + diagnostic_status = Process.run(hook, ["post"], input: IO::Memory.new({"tool_input" => {"file_path" => edited_file}}.to_json), + output: diagnostic_output, error: diagnostic_errors, + env: {"PATH" => path, "TEST_COMMAND_LOG" => log, "TEST_LSP_EXIT" => "1"}) + diagnostic_status.exit_code.should eq(2) + diagnostic_errors.to_s.should contain("lsp violation") + + failed = run_hook.call("stop", "{}", "1", "0") + failed[0].should eq(2) + failed[2].should contain("watch build failed") + File.read(log).should_not contain("compiler build --no-codegen") + + fallback = run_hook.call("stop", "{}", "2", "0") + fallback[0].should eq(0) + File.read(log).should contain("compiler build --no-codegen src/my_app.cr") + + failed_fallback = run_hook.call("stop", "{}", "2", "1") + failed_fallback[0].should eq(2) + failed_fallback[2].should contain("fallback failed") + + run_hook.call("pre", "{}", "0", "2")[0].should eq(0) + + File.rename(compiler, File.join(tools, "acrystal")) + run_hook.call("pre", "{}", "0", "0")[0].should eq(0) + File.rename(File.join(tools, "acrystal"), File.join(tools, "crystal")) + stock_result = run_hook.call("stop", "{}", "0", "0") + stock_result[0].should eq(0) + stock_result[2].should contain("fast rebuilds need crystal-alpha") + + override_compiler = File.join(tools, "custom-alpha") + File.write(override_compiler, "#!/bin/sh\nprintf 'override %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\n") + File.chmod(override_compiler, 0o755) + override_status = Process.run(hook, ["pre"], input: IO::Memory.new("{}"), output: IO::Memory.new, + error: IO::Memory.new, env: {"PATH" => path, "CRYSTAL_ALPHA" => override_compiler, "TEST_COMMAND_LOG" => log}) + override_status.exit_code.should eq(0) + File.read(log).should contain("override watch hold") + end + end +end diff --git a/spec/commands/setup_lsp_spec.cr b/spec/commands/setup_lsp_spec.cr new file mode 100644 index 0000000..5190fd8 --- /dev/null +++ b/spec/commands/setup_lsp_spec.cr @@ -0,0 +1,17 @@ +require "../amber_cli_spec" +require "../../src/amber_cli/commands/setup_lsp" + +describe AmberCLI::Commands::SetupLSPCommand do + it "writes the existing LSP discovery configuration from an Amber project" do + SpecHelper.within_temp_directory do + AmberCLI::Commands::SetupLSPCommand.new("setup:lsp").execute + + config = File.read(".lsp.json") + config.should contain(%("amber")) + config.should contain(%("extensionToLanguage")) + config.should contain(%(".cr")) + File.file?(".claude-plugin/plugin.json").should be_true + File.file?(".amber-lsp.yml").should be_true + end + end +end diff --git a/src/amber_cli.cr b/src/amber_cli.cr index b886e52..14ff2cd 100644 --- a/src/amber_cli.cr +++ b/src/amber_cli.cr @@ -27,6 +27,7 @@ require "./amber_cli/commands/plugin" require "./amber_cli/commands/pipelines" require "./amber_cli/commands/generate" require "./amber_cli/commands/setup_lsp" +require "./amber_cli/commands/setup_agent" backend = Log::IOBackend.new backend.formatter = Log::Formatter.new do |entry, io| @@ -78,6 +79,7 @@ module AmberCLI plugin (pl) Generate application plugins pipelines Show application pipelines and plugs setup:lsp (lsp) Set up Amber LSP for Claude Code integration + setup:agent (agent) Set up Claude Code and Codex feedback hooks Options: --version, -v Show version number diff --git a/src/amber_cli/agent/resolve_compiler_for_agent_loop.cr b/src/amber_cli/agent/resolve_compiler_for_agent_loop.cr new file mode 100644 index 0000000..a9ec9a7 --- /dev/null +++ b/src/amber_cli/agent/resolve_compiler_for_agent_loop.cr @@ -0,0 +1,24 @@ +module AmberCLI::Agent + # Resolves the same compiler order used by generated agent hooks. + class ResolveCompilerForAgentLoop + getter? stock_compiler : Bool = false + + def initialize(@specified_path : String?, @path_lookup : Proc(String, String?)) + end + + def perform : String? + if specified_path = @specified_path + return specified_path unless specified_path.empty? + end + + ["crystal-alpha", "acrystal", "crystal"].each do |command| + if executable = @path_lookup.call(command) + @stock_compiler = command == "crystal" + return executable + end + end + + nil + end + end +end diff --git a/src/amber_cli/commands/new.cr b/src/amber_cli/commands/new.cr index e7e3a2e..04697c6 100644 --- a/src/amber_cli/commands/new.cr +++ b/src/amber_cli/commands/new.cr @@ -197,6 +197,7 @@ module AmberCLI::Commands info " amber generate scaffold Pet name:string:required species:string:required" info " amber database migrate" info " amber watch" + info " Tip: run amber setup:agent to install Claude Code and Codex hooks." info " # Choose -d pg or -d mysql when you need a server database." end diff --git a/src/amber_cli/commands/setup_agent.cr b/src/amber_cli/commands/setup_agent.cr new file mode 100644 index 0000000..3983093 --- /dev/null +++ b/src/amber_cli/commands/setup_agent.cr @@ -0,0 +1,165 @@ +require "json" +require "yaml" +require "../core/base_command" +require "./setup_lsp" + +module AmberCLI::Agent + class HookHandler + include JSON::Serializable + include JSON::Serializable::Unmapped + + property type : String? = nil + property command : String? = nil + + def initialize(@type : String?, @command : String?) + end + end + + class HookGroup + include JSON::Serializable + include JSON::Serializable::Unmapped + + @[JSON::Field(emit_null: false)] + property matcher : String? = nil + property hooks : Array(HookHandler) = [] of HookHandler + + def initialize(@matcher : String?, @hooks : Array(HookHandler)) + end + end + + class HookEvents + include JSON::Serializable + include JSON::Serializable::Unmapped + + @[JSON::Field(key: "PreToolUse")] + property pre_tool_use : Array(HookGroup) = [] of HookGroup + @[JSON::Field(key: "PostToolUse")] + property post_tool_use : Array(HookGroup) = [] of HookGroup + @[JSON::Field(key: "Stop")] + property stop : Array(HookGroup) = [] of HookGroup + + def initialize + end + end + + class HookSettings + include JSON::Serializable + include JSON::Serializable::Unmapped + + property hooks : HookEvents = HookEvents.new + + def initialize + end + end + + # When an agent loop is installed, retain unrelated hook groups and settings. + class MergeAgentHooksIntoSettings + MATCHER = "Edit|Write|MultiEdit|NotebookEdit" + + def initialize(@existing_json : String) + end + + def perform : String + settings = @existing_json.empty? ? HookSettings.new : HookSettings.from_json(@existing_json) + add_hook(settings.hooks.pre_tool_use, MATCHER, "bin/amber-agent-hook pre") + add_hook(settings.hooks.post_tool_use, MATCHER, "bin/amber-agent-hook post") + add_hook(settings.hooks.stop, nil, "bin/amber-agent-hook stop") + settings.to_pretty_json + "\n" + end + + private def add_hook(groups : Array(HookGroup), matcher : String?, command : String) : Nil + group = groups.find { |candidate| candidate.matcher == matcher } + unless group + group = HookGroup.new(matcher, [] of HookHandler) + groups << group + end + return if group.hooks.any? { |handler| handler.type == "command" && handler.command == command } + + group.hooks << HookHandler.new("command", command) + end + end +end + +module AmberCLI::Commands + # Installs the optional Claude Code and Codex feedback loop into an Amber V2 app. + class SetupAgentCommand < AmberCLI::Core::BaseCommand + AGENT_HOOK_SCRIPT = {{ read_file("#{__DIR__}/../templates/agent/amber-agent-hook") }} + DOCUMENT_START = "" + DOCUMENT_END = "" + + def help_description : String + "Set up the Claude Code and Codex agent loop in an Amber V2 project" + end + + def setup_command_options + end + + def execute + main_file = find_project_main_file + SetupLSPCommand.new("setup:lsp").execute + [".claude/settings.json", ".codex/hooks.json"].each do |path| + write_merged_hooks(path) + end + write_hook_script(main_file) + ["CLAUDE.md", "AGENTS.md"].each do |path| + append_agent_loop_instructions(path) + end + success "Amber agent loop installed." + end + + private def find_project_main_file : String + unless File.file?("shard.yml") + raise "Run amber setup:agent from an Amber project with shard.yml" + end + + manifest = YAML.parse(File.read("shard.yml")) + targets = manifest["targets"]?.try(&.as_h?) + main_file = targets.try(&.values.first?).try(&.["main"]?).try(&.as_s?) + unless main_file && main_file.matches?(/\A(?:src\/)?[A-Za-z0-9_\/.-]+\.cr\z/) && !main_file.includes?("..") && File.file?(main_file) + raise "shard.yml must declare an existing targets..main Crystal file" + end + main_file + end + + private def write_merged_hooks(path : String) : Nil + existing_json = File.file?(path) ? File.read(path) : "" + merged_json = AmberCLI::Agent::MergeAgentHooksIntoSettings.new(existing_json).perform + return if existing_json == merged_json + + Dir.mkdir_p(File.dirname(path)) + File.write(path, merged_json) + info "Updated: #{path}" + end + + private def write_hook_script(main_file : String) : Nil + path = "bin/amber-agent-hook" + content = AGENT_HOOK_SCRIPT.sub("__AMBER_MAIN__", main_file) + Dir.mkdir_p("bin") + File.write(path, content) unless File.file?(path) && File.read(path) == content + File.chmod(path, 0o755) + info "Updated: #{path}" + end + + private def append_agent_loop_instructions(path : String) : Nil + content = File.file?(path) ? File.read(path) : "" + return if content.includes?(DOCUMENT_START) && content.includes?(DOCUMENT_END) + raise "Incomplete Amber agent loop marker in #{path}" if content.includes?(DOCUMENT_START) || content.includes?(DOCUMENT_END) + + section = <<-MARKDOWN + #{DOCUMENT_START} + ## Agent loop + + Run `crystal-alpha watch build` after edits. Do not start another watcher. + Use `crystal spec --affected` when available. Format Crystal files with + `crystal-alpha tool format`. The installed hooks hold the watcher during + edits, check each changed file, and build when the agent stops. + #{DOCUMENT_END} + MARKDOWN + prefix = content.empty? ? "" : content.rstrip + "\n\n" + File.write(path, prefix + section) + info "Updated: #{path}" + end + end +end + +AmberCLI::Core::CommandRegistry.register("setup:agent", ["agent"], AmberCLI::Commands::SetupAgentCommand) diff --git a/src/amber_cli/commands/setup_lsp.cr b/src/amber_cli/commands/setup_lsp.cr index 94adcb1..77bcccc 100644 --- a/src/amber_cli/commands/setup_lsp.cr +++ b/src/amber_cli/commands/setup_lsp.cr @@ -1,4 +1,5 @@ require "../core/base_command" +require "../agent/resolve_compiler_for_agent_loop" # The `setup:lsp` command configures the Amber LSP server for Claude Code # integration in an Amber project directory. @@ -126,12 +127,6 @@ module AmberCLI::Commands return Dir.current end - # Check the known development location - dev_path = File.expand_path("~/open_source_coding_projects/amber_cli") - if File.exists?(File.join(dev_path, "src", "amber_lsp.cr")) - return dev_path - end - nil end @@ -145,8 +140,18 @@ module AmberCLI::Commands Dir.mkdir_p(File.join(cli_project_root, "bin")) unless Dir.exists?(File.join(cli_project_root, "bin")) + resolver = AmberCLI::Agent::ResolveCompilerForAgentLoop.new( + ENV["CRYSTAL_ALPHA"]?, ->(command : String) { Process.find_executable(command) } + ) + compiler = resolver.perform + unless compiler + error "No Crystal compiler found (CRYSTAL_ALPHA, crystal-alpha, acrystal, crystal)" + exit(1) + end + warning "Fast rebuilds need crystal-alpha; using stock crystal." if resolver.stock_compiler? + process = Process.run( - "crystal", + compiler, ["build", source_path, "-o", binary_path, "--release"], output: Process::Redirect::Inherit, error: Process::Redirect::Inherit diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook new file mode 100644 index 0000000..d5c2d50 --- /dev/null +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -0,0 +1,144 @@ +#!/bin/sh +set -u + +project_root=$(CDPATH='' cd "$(dirname "$0")/.." && pwd -P) || exit 2 +cd "$project_root" || exit 2 + +compiler_is_stock=0 +if [ -n "${CRYSTAL_ALPHA:-}" ]; then + compiler=$CRYSTAL_ALPHA +elif command -v crystal-alpha >/dev/null 2>&1; then + compiler=crystal-alpha +elif command -v acrystal >/dev/null 2>&1; then + compiler=acrystal +elif command -v crystal >/dev/null 2>&1; then + compiler=crystal + compiler_is_stock=1 + echo 'amber-agent-hook: fast rebuilds need crystal-alpha; using stock crystal' >&2 +else + echo 'amber-agent-hook: no Crystal compiler found' >&2 + exit 2 +fi + +case "${1:-}" in + pre) + if [ "$compiler_is_stock" -eq 0 ]; then + "$compiler" watch hold + status=$? + [ "$status" -eq 2 ] && exit 0 + exit "$status" + fi + ;; + post) + # Claude supplies file_path; Codex supplies an apply_patch command. + # Parse JSON strings and patch headers without another runtime. + edited_files=$(awk ' + function string_value( character, escaped, result) { + result = ""; escaped = 0 + while (position <= length(payload)) { + character = substr(payload, position++, 1) + if (escaped) { + if (character == "n") result = result "\n" + else if (character == "r") result = result "\r" + else if (character == "t") result = result "\t" + else if (character == "u") return "" + else result = result character + escaped = 0; continue + } + if (character == "\\") { escaped = 1; continue } + if (character == "\"") return result + result = result character + } + return "" + } + { payload = payload $0 "\n" } + END { + position = 1 + while (position <= length(payload)) { + if (substr(payload, position++, 1) != "\"") continue + key = string_value() + while (substr(payload, position, 1) ~ /[[:space:]]/) position++ + if (substr(payload, position++, 1) != ":") continue + while (substr(payload, position, 1) ~ /[[:space:]]/) position++ + if (substr(payload, position++, 1) != "\"") continue + value = string_value() + if (key == "file_path") direct_path = value + else if (key == "path" && direct_path == "") direct_path = value + else if (key == "tool_name") tool_name = value + else if (key == "command") patch = value + } + if (tool_name == "apply_patch" && index(patch, "*** Begin Patch") > 0) { + count = split(patch, lines, "\n") + for (line_number = 1; line_number <= count; line_number++) { + line = lines[line_number] + if (line ~ /^\*\*\* (Update|Add) File: /) { + sub(/^\*\*\* (Update|Add) File: /, "", line) + sub(/\r$/, "", line) + if (!seen[line]++) print line + } + } + } else if (direct_path != "") { + print direct_path + } + } + ') + [ -n "$edited_files" ] || exit 0 + printf '%s\n' "$edited_files" | while IFS= read -r edited_file; do + case "$edited_file" in + *.cr) ;; + *) continue ;; + esac + case "$edited_file" in + /*) ;; + *) edited_file=$project_root/$edited_file ;; + esac + [ -f "$edited_file" ] || continue + [ -L "$edited_file" ] && continue + edited_directory=$(CDPATH='' cd "$(dirname "$edited_file")" && pwd -P) || continue + case "$edited_directory" in + "$project_root"|"$project_root"/*) ;; + *) continue ;; + esac + "$compiler" tool format "$edited_file" || exit 2 + if command -v amber-lsp >/dev/null 2>&1; then + diagnostics=$(amber-lsp --check "$edited_file" 2>&1) + diagnostic_status=$? + if [ "$diagnostic_status" -ne 0 ]; then + [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" >&2 + exit 2 + fi + [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" + else + echo 'amber-agent-hook: amber-lsp is required for file diagnostics' >&2 + exit 2 + fi + done + exit $? + ;; + stop) + if [ "$compiler_is_stock" -eq 0 ]; then + "$compiler" watch release >/dev/null 2>&1 || : + output=$("$compiler" watch build 2>&1) + status=$? + if [ "$status" -eq 0 ]; then + [ -z "$output" ] || printf '%s\n' "$output" + exit 0 + fi + if [ "$status" -ne 2 ]; then + [ -z "$output" ] || printf '%s\n' "$output" >&2 + exit 2 + fi + fi + output=$("$compiler" build --no-codegen '__AMBER_MAIN__' 2>&1) + status=$? + if [ "$status" -ne 0 ]; then + [ -z "$output" ] || printf '%s\n' "$output" >&2 + exit 2 + fi + [ -z "$output" ] || printf '%s\n' "$output" + ;; + *) + echo 'Usage: bin/amber-agent-hook pre|post|stop' >&2 + exit 2 + ;; +esac diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 472f055..1fa26ae 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -24,5 +24,15 @@ require "./amber_lsp/configuration" require "./amber_lsp/analyzer" require "./amber_lsp/controller" require "./amber_lsp/server" +require "./amber_lsp/check_file_for_diagnostics" -AmberLSP::Server.new(STDIN, STDOUT).run +if ARGV[0]? == "--check" + file_path = ARGV[1]? + unless file_path + STDERR.puts "Usage: amber-lsp --check FILE.cr" + exit(2) + end + exit(AmberLSP::CheckFileForDiagnostics.new(file_path).perform) +else + AmberLSP::Server.new(STDIN, STDOUT).run +end diff --git a/src/amber_lsp/check_file_for_diagnostics.cr b/src/amber_lsp/check_file_for_diagnostics.cr new file mode 100644 index 0000000..c962fd0 --- /dev/null +++ b/src/amber_lsp/check_file_for_diagnostics.cr @@ -0,0 +1,26 @@ +module AmberLSP + class CheckFileForDiagnostics + def initialize(@file_path : String, @output : IO = STDOUT, @error : IO = STDERR) + end + + def perform : Int32 + path = File.expand_path(@file_path) + unless File.file?(path) + @error.puts "amber-lsp: file not found: #{path}" + return 2 + end + + analyzer = Analyzer.new + analyzer.configure(ProjectContext.detect(Dir.current)) + diagnostics = analyzer.analyze(path, File.read(path)) + diagnostics.each do |diagnostic| + line = diagnostic.range.start.line + 1 + @output.puts "#{path}:#{line}: #{diagnostic.severity.to_s.downcase}: #{diagnostic.code}: #{diagnostic.message}" + end + diagnostics.any? { |diagnostic| diagnostic.severity == Rules::Severity::Error } ? 1 : 0 + rescue ex : File::Error | YAML::ParseException + @error.puts "amber-lsp: #{ex.message}" + 2 + end + end +end From 768233e2e4898ba2e769f8c1c0f33d5486d595cd Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Sat, 3 Oct 2026 12:17:39 -0400 Subject: [PATCH 05/69] fix: support installed Crystal agent hooks --- spec/commands/setup_agent_spec.cr | 154 +++++++++++++++++- src/amber_cli/commands/setup_agent.cr | 34 +++- .../templates/agent/amber-agent-hook | 94 ++++++++++- 3 files changed, 268 insertions(+), 14 deletions(-) diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr index 6e5470b..ef35252 100644 --- a/spec/commands/setup_agent_spec.cr +++ b/spec/commands/setup_agent_spec.cr @@ -1,6 +1,14 @@ require "../amber_cli_spec" require "../../src/amber_cli/commands/setup_agent" +class RecordSetupAgentMessages < AmberCLI::Commands::SetupAgentCommand + getter list_of_info_messages : Array(String) = [] of String + + protected def info(message : String) + @list_of_info_messages << message + end +end + describe "amber setup:agent" do it "registers the setup command and short alias" do AmberCLI::Core::CommandRegistry.find_command("setup:agent").should_not be_nil @@ -56,7 +64,7 @@ describe "amber setup:agent" do File.write(".claude/settings.json", %({"permissions":{"allow":["Read"]}})) File.write(".codex/hooks.json", %({"description":"keep this","hooks":{"SessionStart":[]}})) - command = AmberCLI::Commands::SetupAgentCommand.new("setup:agent") + command = RecordSetupAgentMessages.new("setup:agent") command.execute first_claude = File.read(".claude/settings.json") first_codex = File.read(".codex/hooks.json") @@ -72,9 +80,11 @@ describe "amber setup:agent" do File.read("CLAUDE.md").should eq(first_instructions) File.read("AGENTS.md").scan(/amber-agent-loop:start/).size.should eq(1) first_instructions.should contain("# Existing Claude instructions") + first_instructions.should contain("crystal-alpha spec --affected") first_script.should contain("build --no-codegen 'src/custom_entry.cr'") File.file?(".lsp.json").should be_true File.info("bin/amber-agent-hook").permissions.to_i.&(0o111).should_not eq(0) + command.list_of_info_messages.count("Updated: bin/amber-agent-hook").should eq(1) end end @@ -95,6 +105,7 @@ describe "amber setup:agent" do File.write(lsp, "#!/bin/sh\nprintf 'lsp %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\nif [ \"${TEST_LSP_EXIT:-0}\" -ne 0 ]; then\n echo 'lsp violation'\n exit \"$TEST_LSP_EXIT\"\nfi\n") File.chmod(compiler, 0o755) File.chmod(lsp, 0o755) + File.write(".lsp.json", {"amber" => {"command" => lsp}}.to_pretty_json + "\n") path = "#{tools}:/usr/bin:/bin" hook = File.join(project, "bin/amber-agent-hook") @@ -172,4 +183,145 @@ describe "amber setup:agent" do File.read(log).should contain("override watch hold") end end + + it "uses the no-targets shard name as the main file" do + SpecHelper.within_temp_directory do + Dir.mkdir_p("src") + File.write("shard.yml", "name: fallback_app\nversion: 0.1.0\n") + File.write("src/fallback_app.cr", "puts :ok\n") + + AmberCLI::Commands::SetupAgentCommand.new("setup:agent").execute + + File.read("bin/amber-agent-hook").should contain("build --no-codegen 'src/fallback_app.cr'") + end + end + + it "corrects older marked agent instructions on rerun" do + SpecHelper.within_temp_directory do + Dir.mkdir_p("src") + File.write("shard.yml", "name: my_app\n") + File.write("src/my_app.cr", "puts :ok\n") + older_instructions = "# Keep this\n\n\nUse `crystal spec --affected` when available.\n\n" + File.write("CLAUDE.md", older_instructions) + File.write("AGENTS.md", older_instructions) + + command = AmberCLI::Commands::SetupAgentCommand.new("setup:agent") + command.execute + first_instructions = File.read("CLAUDE.md") + first_instructions.should contain("# Keep this") + first_instructions.should contain("Use `crystal-alpha spec --affected`") + first_instructions.should_not contain("Use `crystal spec --affected`") + File.read("AGENTS.md").should eq(first_instructions) + + command.execute + File.read("CLAUDE.md").should eq(first_instructions) + end + end + + it "treats the installed alpha without watch coordination as a build-only compiler" do + SpecHelper.within_temp_directory do |project| + Dir.mkdir_p("src") + File.write("shard.yml", "name: my_app\ntargets:\n my_app:\n main: src/my_app.cr\n") + File.write("src/my_app.cr", "puts :ok\n") + AmberCLI::Commands::SetupAgentCommand.new("setup:agent").execute + + tools = File.join(project, "fake-tools") + Dir.mkdir_p(tools) + compiler = File.join(tools, "crystal-alpha") + log = File.join(project, "calls.log") + File.write(compiler, "#!/bin/sh\nprintf 'compiler %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\ncase \"$1\" in\n watch) echo 'Usage: crystal watch [options] [programfile]'; exit 1 ;;\n build) exit 0 ;;\nesac\n") + File.chmod(compiler, 0o755) + hook = File.join(project, "bin/amber-agent-hook") + environment = {"PATH" => "#{tools}:/usr/bin:/bin", "TEST_COMMAND_LOG" => log} + + pre_output = IO::Memory.new + pre_errors = IO::Memory.new + pre_status = Process.run(hook, ["pre"], input: IO::Memory.new("{}"), output: pre_output, error: pre_errors, env: environment) + pre_status.exit_code.should eq(0) + pre_output.to_s.should_not contain("Usage:") + + stop_output = IO::Memory.new + stop_errors = IO::Memory.new + stop_status = Process.run(hook, ["stop"], input: IO::Memory.new("{}"), output: stop_output, error: stop_errors, env: environment) + stop_status.exit_code.should eq(0) + stop_errors.to_s.should_not contain("Usage:") + calls = File.read(log) + calls.scan(/compiler watch status/).size.should eq(2) + calls.should_not contain("compiler watch hold") + calls.should_not contain("compiler watch release") + calls.should_not contain("compiler watch build") + calls.should contain("compiler build --no-codegen src/my_app.cr") + end + end + + it "formats without an LSP and finds a project LSP when configured one is absent" do + SpecHelper.within_temp_directory do |project| + Dir.mkdir_p("src") + File.write("shard.yml", "name: my_app\ntargets:\n my_app:\n main: src/my_app.cr\n") + File.write("src/my_app.cr", "puts :ok\n") + AmberCLI::Commands::SetupAgentCommand.new("setup:agent").execute + File.write(".lsp.json", {"amber" => {"command" => "/missing/amber-lsp"}}.to_pretty_json + "\n") + + tools = File.join(project, "fake-tools") + Dir.mkdir_p(tools) + compiler = File.join(tools, "crystal-alpha") + log = File.join(project, "calls.log") + File.write(compiler, "#!/bin/sh\nprintf 'compiler %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\nif [ \"${TEST_FORMAT_EXIT:-0}\" -ne 0 ]; then\n echo 'format failed'\n exit \"$TEST_FORMAT_EXIT\"\nfi\n") + File.chmod(compiler, 0o755) + hook = File.join(project, "bin/amber-agent-hook") + payload = {"tool_input" => {"file_path" => "src/my_app.cr"}}.to_json + File.write("src/other.cr", "puts :other\n") + patch = "*** Begin Patch\n*** Update File: src/my_app.cr\n*** Update File: src/other.cr\n*** End Patch" + multiple_files_payload = {"tool_name" => "apply_patch", "tool_input" => {"command" => patch}}.to_json + environment = {"PATH" => "#{tools}:/usr/bin:/bin", "TEST_COMMAND_LOG" => log} + + errors = IO::Memory.new + missing_status = Process.run(hook, ["post"], input: IO::Memory.new(multiple_files_payload), output: IO::Memory.new, error: errors, env: environment) + missing_status.exit_code.should eq(0) + errors.to_s.scan(/amber-lsp unavailable/).size.should eq(1) + File.read(log).scan(/compiler tool format /).size.should eq(2) + + project_lsp = File.join(project, "bin/amber-lsp") + File.write(project_lsp, "#!/bin/sh\nprintf 'project-lsp %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\n") + File.chmod(project_lsp, 0o755) + errors = IO::Memory.new + project_status = Process.run(hook, ["post"], input: IO::Memory.new(payload), output: IO::Memory.new, error: errors, env: environment) + project_status.exit_code.should eq(0) + errors.to_s.should be_empty + File.read(log).should contain("project-lsp --check ") + + path_lsp = File.join(tools, "amber-lsp") + File.write(path_lsp, "#!/bin/sh\nprintf 'path-lsp %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\n") + File.chmod(path_lsp, 0o755) + File.write(".lsp.json", {"amber" => {"command" => "amber-lsp"}}.to_pretty_json + "\n") + project_calls_before = File.read(log).scan(/project-lsp --check /).size + project_priority = Process.run(hook, ["post"], input: IO::Memory.new(payload), output: IO::Memory.new, + error: IO::Memory.new, env: environment) + project_priority.exit_code.should eq(0) + File.read(log).scan(/project-lsp --check /).size.should eq(project_calls_before + 1) + File.read(log).should_not contain("path-lsp --check ") + + File.delete(project_lsp) + File.write(".lsp.json", {"amber" => {"command" => "/missing/amber-lsp"}}.to_pretty_json + "\n") + path_status = Process.run(hook, ["post"], input: IO::Memory.new(payload), output: IO::Memory.new, + error: IO::Memory.new, env: environment) + path_status.exit_code.should eq(0) + File.read(log).should contain("path-lsp --check ") + + configured_lsp = File.join(tools, "configured-lsp") + File.write(configured_lsp, "#!/bin/sh\nprintf 'configured-lsp %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\n") + File.chmod(configured_lsp, 0o755) + File.write(".lsp.json", {"amber" => {"command" => configured_lsp}}.to_pretty_json + "\n") + configured_status = Process.run(hook, ["post"], input: IO::Memory.new(payload), output: IO::Memory.new, + error: IO::Memory.new, env: environment) + configured_status.exit_code.should eq(0) + File.read(log).should contain("configured-lsp --check ") + + format_errors = IO::Memory.new + failed_format = Process.run(hook, ["post"], input: IO::Memory.new(payload), output: IO::Memory.new, + error: format_errors, env: environment.merge({"TEST_FORMAT_EXIT" => "1"})) + failed_format.exit_code.should eq(2) + format_errors.to_s.should contain("format failed") + end + end end diff --git a/src/amber_cli/commands/setup_agent.cr b/src/amber_cli/commands/setup_agent.cr index 3983093..96b0800 100644 --- a/src/amber_cli/commands/setup_agent.cr +++ b/src/amber_cli/commands/setup_agent.cr @@ -114,9 +114,17 @@ module AmberCLI::Commands manifest = YAML.parse(File.read("shard.yml")) targets = manifest["targets"]?.try(&.as_h?) - main_file = targets.try(&.values.first?).try(&.["main"]?).try(&.as_s?) + first_target = targets.try(&.values.first?) + target_main = first_target.try(&.["main"]?) + main_file = target_main.try(&.as_s?) + + unless first_target + project_name = manifest["name"]?.try(&.as_s?) + main_file = "src/#{project_name}.cr" if project_name + end + unless main_file && main_file.matches?(/\A(?:src\/)?[A-Za-z0-9_\/.-]+\.cr\z/) && !main_file.includes?("..") && File.file?(main_file) - raise "shard.yml must declare an existing targets..main Crystal file" + raise "shard.yml must declare an existing targets..main or src/.cr Crystal file" end main_file end @@ -135,14 +143,28 @@ module AmberCLI::Commands path = "bin/amber-agent-hook" content = AGENT_HOOK_SCRIPT.sub("__AMBER_MAIN__", main_file) Dir.mkdir_p("bin") - File.write(path, content) unless File.file?(path) && File.read(path) == content + if !File.file?(path) || File.read(path) != content + File.write(path, content) + info "Updated: #{path}" + end File.chmod(path, 0o755) - info "Updated: #{path}" end private def append_agent_loop_instructions(path : String) : Nil content = File.file?(path) ? File.read(path) : "" - return if content.includes?(DOCUMENT_START) && content.includes?(DOCUMENT_END) + if content.includes?(DOCUMENT_START) && content.includes?(DOCUMENT_END) + marker_start = content.index(DOCUMENT_START) + marker_end = content.index(DOCUMENT_END) + if marker_start && marker_end && marker_start < marker_end + section = content[marker_start...marker_end] + updated_section = section.sub("Use `crystal spec --affected`", "Use `crystal-alpha spec --affected`") + if updated_section != section + File.write(path, content.sub(section, updated_section)) + info "Updated: #{path}" + end + return + end + end raise "Incomplete Amber agent loop marker in #{path}" if content.includes?(DOCUMENT_START) || content.includes?(DOCUMENT_END) section = <<-MARKDOWN @@ -150,7 +172,7 @@ module AmberCLI::Commands ## Agent loop Run `crystal-alpha watch build` after edits. Do not start another watcher. - Use `crystal spec --affected` when available. Format Crystal files with + Use `crystal-alpha spec --affected` when available. Format Crystal files with `crystal-alpha tool format`. The installed hooks hold the watcher during edits, check each changed file, and build when the agent stops. #{DOCUMENT_END} diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook index d5c2d50..7172f13 100644 --- a/src/amber_cli/templates/agent/amber-agent-hook +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -20,9 +20,78 @@ else exit 2 fi +watch_coordination_is_available() { + [ "$compiler_is_stock" -eq 0 ] || return 1 + "$compiler" watch status >/dev/null 2>&1 + watch_status=$? + [ "$watch_status" -eq 0 ] || [ "$watch_status" -eq 2 ] +} + +find_lsp() { + configured_lsp='' + if [ -f .lsp.json ]; then + configured_lsp=$(awk ' + function string_value( character, escaped, result) { + result = ""; escaped = 0 + while (position <= length(document)) { + character = substr(document, position++, 1) + if (escaped) { + if (character == "u") return "" + result = result character + escaped = 0; continue + } + if (character == "\\") { escaped = 1; continue } + if (character == "\"") return result + result = result character + } + return "" + } + { document = document $0 "\n" } + END { + position = 1 + while (position <= length(document)) { + if (substr(document, position++, 1) != "\"") continue + key = string_value() + while (substr(document, position, 1) ~ /[[:space:]]/) position++ + if (substr(document, position++, 1) != ":") continue + while (substr(document, position, 1) ~ /[[:space:]]/) position++ + if (substr(document, position++, 1) != "\"") continue + value = string_value() + if (key == "command") { print value; exit } + } + } + ' .lsp.json) + fi + + if [ -n "$configured_lsp" ]; then + if [ -x "$configured_lsp" ]; then + printf '%s\n' "$configured_lsp" + return 0 + fi + case "$configured_lsp" in + amber-lsp) ;; + *) + if command -v "$configured_lsp" >/dev/null 2>&1; then + printf '%s\n' "$configured_lsp" + return 0 + fi + ;; + esac + fi + if [ -x bin/amber-lsp ]; then + printf '%s\n' bin/amber-lsp + return 0 + fi + if command -v amber-lsp >/dev/null 2>&1; then + printf '%s\n' amber-lsp + return 0 + fi + return 1 +} + case "${1:-}" in pre) - if [ "$compiler_is_stock" -eq 0 ]; then + if watch_coordination_is_available; then "$compiler" watch hold status=$? [ "$status" -eq 2 ] && exit 0 @@ -83,6 +152,7 @@ case "${1:-}" in } ') [ -n "$edited_files" ] || exit 0 + lsp=$(find_lsp) || lsp='' printf '%s\n' "$edited_files" | while IFS= read -r edited_file; do case "$edited_file" in *.cr) ;; @@ -99,9 +169,15 @@ case "${1:-}" in "$project_root"|"$project_root"/*) ;; *) continue ;; esac - "$compiler" tool format "$edited_file" || exit 2 - if command -v amber-lsp >/dev/null 2>&1; then - diagnostics=$(amber-lsp --check "$edited_file" 2>&1) + format_output=$("$compiler" tool format "$edited_file" 2>&1) + format_status=$? + if [ "$format_status" -ne 0 ]; then + [ -z "$format_output" ] || printf '%s\n' "$format_output" >&2 + exit 2 + fi + [ -z "$format_output" ] || printf '%s\n' "$format_output" + if [ -n "$lsp" ]; then + diagnostics=$("$lsp" --check "$edited_file" 2>&1) diagnostic_status=$? if [ "$diagnostic_status" -ne 0 ]; then [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" >&2 @@ -109,14 +185,16 @@ case "${1:-}" in fi [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" else - echo 'amber-agent-hook: amber-lsp is required for file diagnostics' >&2 - exit 2 + if [ "${warned_missing_lsp:-0}" -eq 0 ]; then + echo 'amber-agent-hook: amber-lsp unavailable; skipping file diagnostics' >&2 + warned_missing_lsp=1 + fi fi done exit $? ;; stop) - if [ "$compiler_is_stock" -eq 0 ]; then + if watch_coordination_is_available; then "$compiler" watch release >/dev/null 2>&1 || : output=$("$compiler" watch build 2>&1) status=$? @@ -128,6 +206,8 @@ case "${1:-}" in [ -z "$output" ] || printf '%s\n' "$output" >&2 exit 2 fi + elif [ "$compiler_is_stock" -eq 0 ]; then + echo 'amber-agent-hook: watch coordination unavailable; using direct build' >&2 fi output=$("$compiler" build --no-codegen '__AMBER_MAIN__' 2>&1) status=$? From 99bf975f3e18f75f6560364f89cebc3f1f3d6127 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Sun, 4 Oct 2026 13:49:04 -0400 Subject: [PATCH 06/69] feat(onboarding): generate hash-locked web apps with Minecart --- .github/workflows/build.yml | 5 +- .github/workflows/ci.yml | 15 +++- .github/workflows/docs.yml | 5 +- .github/workflows/platform-compile.yml | 11 ++- .github/workflows/release.yml | 47 +++++++------ .minecart-policy.yml | 4 ++ README.md | 53 ++++++++------ RELEASE_SETUP.md | 76 ++++++++++---------- docs/BETA_WEB_APP.md | 14 ++-- docs/RELEASE_CHECKLIST.md | 30 ++++---- scripts/build_release.sh | 27 +++++--- scripts/check_beta_contract.sh | 11 +-- scripts/generate_docs.sh | 4 +- scripts/install_pinned_minecart.ps1 | 34 +++++++++ scripts/install_pinned_minecart.sh | 25 +++++++ scripts/smoke_generated_web.ps1 | 23 ++++++- scripts/smoke_generated_web.sh | 39 +++++------ shard.lock | 10 +++ shard.yml | 14 ++-- spec/commands/new_command_spec.cr | 38 +++++----- src/amber_cli.cr | 2 +- src/amber_cli/commands/new.cr | 84 ++++++++++++++++++----- src/amber_cli/templates/app/shard.yml.ecr | 2 +- 23 files changed, 378 insertions(+), 195 deletions(-) create mode 100644 .minecart-policy.yml create mode 100644 scripts/install_pinned_minecart.ps1 create mode 100755 scripts/install_pinned_minecart.sh diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index de5b5af..ab241c9 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -51,8 +51,11 @@ jobs: restore-keys: | ${{ runner.os }}-shards- + - name: Build pinned Minecart + run: scripts/install_pinned_minecart.sh + - name: Install dependencies - run: shards install + run: minecart install --frozen - name: Build LSP binary for integration specs run: | diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 476cbd2..cf89f2d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -43,8 +43,11 @@ jobs: restore-keys: | ${{ runner.os }}-shards- + - name: Build pinned Minecart + run: scripts/install_pinned_minecart.sh + - name: Install dependencies - run: shards install + run: minecart install --frozen - name: Check Amber V2 beta contract run: scripts/check_beta_contract.sh @@ -120,8 +123,11 @@ jobs: restore-keys: | ${{ runner.os }}-shards- + - name: Build pinned Minecart + run: scripts/install_pinned_minecart.sh + - name: Install dependencies - run: shards install + run: minecart install --frozen - name: Test CLI functionality run: | @@ -155,8 +161,11 @@ jobs: with: crystal: latest + - name: Build pinned Minecart + run: scripts/install_pinned_minecart.sh + - name: Install dependencies - run: shards install + run: minecart install --frozen - name: Run integration tests run: | diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 910180d..14968a6 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -39,8 +39,11 @@ jobs: with: crystal: latest + - name: Build pinned Minecart + run: scripts/install_pinned_minecart.sh + - name: Install dependencies - run: shards install + run: minecart install --frozen - name: Generate documentation run: | diff --git a/.github/workflows/platform-compile.yml b/.github/workflows/platform-compile.yml index ae8005c..229d2f2 100644 --- a/.github/workflows/platform-compile.yml +++ b/.github/workflows/platform-compile.yml @@ -26,8 +26,11 @@ jobs: with: crystal: latest + - name: Build pinned Minecart + run: scripts/install_pinned_minecart.sh + - name: Install CLI dependencies - run: shards install + run: minecart install --frozen - name: Build Amber CLI run: crystal build src/amber_cli.cr --no-debug -o amber @@ -62,8 +65,12 @@ jobs: "CRYSTAL_LIBRARY_PATH=$env:VCPKG_INSTALLATION_ROOT\installed\x64-windows\lib;$crystalLibraryPath" >> $env:GITHUB_ENV "$env:VCPKG_INSTALLATION_ROOT\installed\x64-windows\bin" >> $env:GITHUB_PATH + - name: Build pinned Minecart + shell: pwsh + run: scripts/install_pinned_minecart.ps1 + - name: Install CLI dependencies - run: shards install + run: minecart install --frozen - name: Build Amber CLI run: crystal build src/amber_cli.cr --no-debug -o amber.exe diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index bac1458..039548f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -64,13 +64,11 @@ jobs: test "v$shard_version" = "$requested_ref" fi + - name: Build pinned Minecart + run: scripts/install_pinned_minecart.sh + - name: Install dependencies - run: | - if [ -f shard.lock ]; then - shards install --production - else - shards install - fi + run: minecart install --production - name: Build binaries (Linux) if: startsWith(matrix.target, 'linux-') @@ -115,7 +113,7 @@ jobs: else shasum -a 256 amber_cli-${{ matrix.target }}.tar.gz > amber_cli-${{ matrix.target }}.tar.gz.sha256 fi - echo "sha256=$(cat amber_cli-${{ matrix.target }}.tar.gz.sha256 | cut -d' ' -f1)" >> $GITHUB_OUTPUT + echo "sha256=$(cat amber_cli-${{ matrix.target }}.tar.gz.sha256 | cut -d' ' -f1)" >> "$GITHUB_OUTPUT" - name: Upload artifact uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 @@ -132,6 +130,24 @@ jobs: if: github.event_name == 'release' steps: + - name: Checkout release source + uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6 + with: + ref: ${{ github.event.release.tag_name }} + + - name: Package checksummed source archive + run: | + version="$(awk '/^version:/ { print $2; exit }' shard.yml)" + test "v$version" = "${{ github.event.release.tag_name }}" + mkdir -p dist + git archive --format=tar --prefix="amber_cli-${version}/" HEAD | gzip -n -9 > "dist/amber_cli-source-${version}.tar.gz" + cd dist + if command -v sha256sum >/dev/null 2>&1; then + sha256sum "amber_cli-source-${version}.tar.gz" > "amber_cli-source-${version}.tar.gz.sha256" + else + shasum -a 256 "amber_cli-source-${version}.tar.gz" > "amber_cli-source-${version}.tar.gz.sha256" + fi + - name: Download all artifacts uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 with: @@ -166,22 +182,9 @@ jobs: artifacts/amber_cli-linux-x86_64/amber_cli-linux-x86_64.tar.gz.sha256 artifacts/amber_cli-linux-arm64/amber_cli-linux-arm64.tar.gz artifacts/amber_cli-linux-arm64/amber_cli-linux-arm64.tar.gz.sha256 + dist/amber_cli-source-2.0.7.tar.gz + dist/amber_cli-source-2.0.7.tar.gz.sha256 tag_name: ${{ github.event.release.tag_name }} fail_on_unmatched_files: true env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - notify-homebrew: - name: Notify Homebrew Tap - needs: upload-assets - runs-on: ubuntu-latest - if: github.event_name == 'release' - - steps: - - name: Dispatch to underscore tap (homebrew-amber_cli) - uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4 - with: - token: ${{ secrets.HOMEBREW_TAP_TOKEN }} - repository: amberframework/homebrew-amber_cli - event-type: release-published - client-payload: '{"version": "${{ github.event.release.tag_name }}"}' diff --git a/.minecart-policy.yml b/.minecart-policy.yml new file mode 100644 index 0000000..45005f3 --- /dev/null +++ b/.minecart-policy.yml @@ -0,0 +1,4 @@ +version: 1 +rules: + dependencies: + require_exact: true diff --git a/README.md b/README.md index 1985bec..7b6f30e 100644 --- a/README.md +++ b/README.md @@ -3,38 +3,44 @@ [![GitHub release](https://img.shields.io/github/release/amberframework/amber_cli.svg)](https://github.com/amberframework/amber_cli/releases) [![Docs](https://img.shields.io/badge/docs-available-brightgreen.svg)](https://amberframework.github.io/amber_cli/) -Amber CLI is the standalone command-line companion for Amber V2. CLI `2.0.6` +Amber CLI is the standalone command-line companion for Amber V2. CLI `2.0.7` creates the supported Amber `2.0.0-beta.5` ECR web application and includes development, generator, database, and LSP tooling. Amber V2 is a beta. The release-gated path is a web application on Apple -Silicon macOS, x86_64 Linux, or ARM64 Linux. Windows x86-64 generated-app -compilation is checked in CI for compatibility, but Windows does not block this -beta release. See [Generator support](docs/GENERATOR_SUPPORT.md) before relying +Silicon macOS, x86_64 Linux, or ARM64 Linux. Windows x86-64 passes the complete +generated-app smoke in CI, but has no release archive yet. See +[Generator support](docs/GENERATOR_SUPPORT.md) before relying on authentication, API-resource, or native output. ## Install -Prerequisites: Crystal 1.20 or newer (but earlier than 2.0), `shards`, and Git. +Prerequisites: Homebrew 7 or newer and Git. The formula installs Crystal, +Minecart, and the native libraries used by the default web app. ### Homebrew on macOS or Linux -The tap and formula names contain underscores; the installed executable is -`amber`: +Run `brew trust --tap amberframework/amber_cli`, then follow the +[installation guide](https://amberframework.org/docs/v2/getting-started/installation/) +to fetch and pin the reviewed tap commit. With that commit checked out, run: ```bash -brew install amberframework/amber_cli/amber_cli +HOMEBREW_NO_AUTO_UPDATE=1 brew install amberframework/amber_cli/amber_cli amber --version +minecart --version ``` -The fully qualified command follows Homebrew's tap-trust model and trusts only -the `amber_cli` formula. +The single install command brings `amber`, `amber-lsp`, Minecart, Crystal, and +SQLite/OpenSSL/PostgreSQL/MySQL client libraries. The tap commit fixes formula +content and source-archive SHA-256 values. Homebrew-core dependency versions +are selected by the installed core catalog, not pinned by this tap. ### Direct release archive -CLI `2.0.6` publishes `darwin-arm64`, `linux-x86_64`, and `linux-arm64` -archives. Windows x86-64 is compiled in CI but does not yet have a release -archive. +The published CLI `2.0.6` has `darwin-arm64`, `linux-x86_64`, and +`linux-arm64` binary archives. A direct archive does not install Minecart or +the native dependencies, so Homebrew is the beginner path. Windows x86-64 has +no release archive. ```bash version=v2.0.6 @@ -55,16 +61,20 @@ command with `sudo` if `/usr/local/bin` is not writable. ## Create and verify a web app ```bash -amber new my_app --type web +amber new my_app --type web -y cd my_app +minecart install --frozen amber assets check crystal spec crystal build src/my_app.cr -o bin/my_app amber watch ``` -`amber new` installs shards by default and compiles the starter assets. Pass -`--no-deps` when an offline or CI workflow needs to run `shards install` later. +`amber new` runs Minecart by default. It writes exact root dependency pins, +a `shard.lock` with Git tree checksums, `.minecart-policy.yml`, and `.claude/` +assistant files. Commit those files. With `--no-deps`, the first Minecart +install and assistant setup are deliberately deferred; run `minecart install +--strict-pinning` and `minecart assistant init` before frozen installs. `amber watch` recompiles assets before the application whenever an ECR template or a file under `app/assets/` changes. Open . @@ -121,6 +131,8 @@ Create the first complete resource and its database table: ```bash amber generate scaffold Pet name:string:required species:string:required adopted:bool amber database migrate +AMBER_ENV=test amber database migrate +crystal spec amber watch ``` @@ -151,8 +163,7 @@ Run `amber --help` or `amber COMMAND --help` for command syntax. The detailed ## Update and troubleshoot ```bash -brew update -brew upgrade amber_cli +HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade amberframework/amber_cli/amber_cli type -a amber amber --version ``` @@ -180,9 +191,9 @@ See the [LSP setup guide](https://github.com/amberframework/amber/blob/v2.0.0-be ## Contributing ```bash -shards install -crystal tool format --check src spec -crystal spec +minecart install --frozen +crystal-alpha tool format --check src spec +crystal-alpha spec ``` See [CONTRIBUTING.md](CONTRIBUTING.md) for the project workflow. diff --git a/RELEASE_SETUP.md b/RELEASE_SETUP.md index 5c0d5f0..dc76295 100644 --- a/RELEASE_SETUP.md +++ b/RELEASE_SETUP.md @@ -4,27 +4,29 @@ This guide documents the current Amber CLI release path and the checks we expect ## What "Done" Looks Like -A successful release means all of the following happen without manual file editing: +A successful release means all of the following have been reviewed and run: 1. A published GitHub release in `amberframework/amber_cli` builds macOS and Linux binaries. -2. The workflow uploads release archives and checksum files to that release. -3. The workflow dispatches `amberframework/homebrew-amber_cli`. -4. The tap rewrites `Formula/amber_cli.rb` with the new version and checksums. -5. The tap explicitly dispatches its install validation after the bot pushes the - formula. This is required because pushes made by `GITHUB_TOKEN` do not start - push-triggered workflows. -6. On Apple Silicon macOS, x86_64 Linux, and ARM64 Linux, validation proves a clean +2. The workflow uploads binary archives and checksum files. A source archive + from the reviewed commit is uploaded with its recorded SHA-256 for Homebrew. +3. The reviewed tap commit contains the Amber and Minecart source archive URLs + and SHA-256 values. It is pushed only after both archives are available. +4. Tap install validation passes on supported Homebrew platforms. +5. On Apple Silicon macOS, x86_64 Linux, and ARM64 Linux, validation proves a clean machine can: - `brew install amberframework/amber_cli/amber_cli` - `brew test amber_cli` - create the ECR web template with `amber new smoke_app --type web` - - install shards, build and verify assets, run specs, and build the generated app + - use Minecart to write and verify the app lock, build and verify assets, run + specs, and build the generated app - launch the built app and request `/` plus its manifest-rendered fingerprinted CSS, JavaScript, image, and favicon URLs -The fully qualified Homebrew command trusts only the `amber_cli` formula under -Homebrew's third-party tap trust model. Do not replace it with a separate -`brew tap` step in public installation instructions. +The assistant runs `brew trust --tap amberframework/amber_cli`, fetches the +tap, checks out the reviewed full commit, and disables Homebrew auto-update +for the single `brew install` command. Trust covers the tap; the commit fixes +the formula content. Homebrew-core dependencies still follow the local core +catalog rather than exact versions pinned by this tap. If any one of those steps is red, the release is not ready to announce. @@ -45,18 +47,13 @@ Use the repository PR template for this so release context stays attached to the - [`.github/workflows/release.yml`](.github/workflows/release.yml) - [`scripts/build_release.sh`](scripts/build_release.sh) - `amberframework/homebrew-amber_cli` - - `Formula/amber_cli.rb` - - `.github/workflows/update-formula.yml` + - `Formula/amber_cli.rb` and `Formula/minecart.rb` - `.github/workflows/validate-install.yml` ## Required Secrets -`amberframework/amber_cli` needs a `HOMEBREW_TAP_TOKEN` secret that can dispatch workflows in `amberframework/homebrew-amber_cli`. - -Recommended scopes for a classic PAT: - -- `repo` -- `workflow` +No cross-repository token is needed. Formula updates are reviewed and committed +with the exact source archive SHA-256 values after the release assets exist. ## Release Flow @@ -106,8 +103,11 @@ Publishing the release triggers the automated flow: 1. build macOS and Linux binaries 2. upload archives and checksums to the release -3. dispatch the Homebrew tap update -4. run the tap smoke test on macOS and Linux +3. publish the binary archive checksums alongside the archives + +Upload the prepared `amber_cli-source-2.0.7.tar.gz` archive without rebuilding +it. Recompute its SHA-256 and compare it with `Formula/amber_cli.rb` before +pushing the tap commit. Then run the tap smoke on macOS and Linux. ## CI Gates To Check @@ -119,13 +119,6 @@ In `amberframework/amber_cli`, the release workflow must be green for: - `Build linux-x86_64` - `Build linux-arm64` - `Upload Release Assets` -- `Notify Homebrew Tap` - -### Tap update - -In `amberframework/homebrew-amber_cli`, the formula update workflow must be green for: - -- `Update Formula` ### Tap install smoke @@ -137,11 +130,12 @@ In `amberframework/homebrew-amber_cli`, the install smoke workflow must be green That workflow explicitly runs: ```bash -brew install amberframework/amber_cli/amber_cli +brew trust --tap amberframework/amber_cli +HOMEBREW_NO_AUTO_UPDATE=1 brew install amberframework/amber_cli/amber_cli brew test amber_cli -amber new smoke_app --type web -y --no-deps +amber new smoke_app --type web -y cd smoke_app -shards install +minecart install --frozen amber assets build amber assets check crystal spec @@ -158,10 +152,10 @@ also rejects binaries linked to `openssl@1.1`. If the tap update fails after a release: 1. Download the release assets and checksum files from GitHub. -2. Update `Formula/amber_cli.rb` in `amberframework/homebrew-amber_cli`. -3. Commit and push to `main`. -4. Explicitly dispatch `.github/workflows/validate-install.yml`; a bot-token - push alone will not trigger it. +2. Compare the source asset hash with `Formula/amber_cli.rb` and the Minecart + source asset hash with `Formula/minecart.rb`. +3. Fix any mismatch in the review branch; never skip verification. +4. Push the reviewed tap commit only after validation passes. If the release build fails before the tap update: @@ -171,9 +165,9 @@ If the release build fails before the tap update: ## Current Packaging Direction -The Homebrew tap and matching release archives are the supported install paths -for Apple Silicon macOS, x86_64 Linux, and ARM64 Linux today. Windows x86-64 is -compiled in CI as a compatibility check but does not gate this beta and does not -yet have a release archive. +The Homebrew tap is the one-install beginner path after trust and tap pinning. +Windows x86-64 passes the generated-app CI gate but has no release archive. -For eventual `homebrew/core` inclusion, we should plan for a source-building formula and a clean `brew audit --new --formula amber_cli` story. The current tap keeps release onboarding fast, while the source-build path is the more likely route for upstream Homebrew acceptance. +The tap builds both Minecart and Amber CLI from hash-checked source archives. +For eventual `homebrew/core` inclusion, validate the formula against core's +additional audit rules. diff --git a/docs/BETA_WEB_APP.md b/docs/BETA_WEB_APP.md index c668d36..333b226 100644 --- a/docs/BETA_WEB_APP.md +++ b/docs/BETA_WEB_APP.md @@ -1,28 +1,30 @@ # Amber V2 Beta Web App -This guide is the consumer smoke test for Amber CLI `2.0.6` and Amber +This guide is the consumer smoke test for Amber CLI `2.0.7` and Amber `2.0.0-beta.5`. It is expected to pass on Apple Silicon macOS, x86_64 Linux, -and ARM64 Linux. Windows x86-64 compilation is checked in CI. +and ARM64 Linux. Windows x86-64 runs the complete generated-app smoke in CI. ## 1. Verify the toolchain ```bash crystal --version -shards --version +minecart --version amber --version ``` -Crystal must be at least 1.20 and earlier than 2.0. Amber CLI must be 2.0.6 or +Crystal must be at least 1.20 and earlier than 2.0. Amber CLI must be 2.0.7 or newer. ## 2. Generate the web app ```bash -amber new amber_beta_smoke --type web +amber new amber_beta_smoke --type web -y cd amber_beta_smoke +minecart install --frozen ``` -The command installs dependencies unless `--no-deps` is passed. Inspect the +The command uses Minecart to install dependencies, write a checksummed +`shard.lock`, and add `.claude/` assistant files unless `--no-deps` is passed. Inspect the contract before continuing: ```bash diff --git a/docs/RELEASE_CHECKLIST.md b/docs/RELEASE_CHECKLIST.md index b6ca695..6c176cc 100644 --- a/docs/RELEASE_CHECKLIST.md +++ b/docs/RELEASE_CHECKLIST.md @@ -1,7 +1,7 @@ # Amber V2 Beta Release Checklist Release order matters because generated apps pin a framework version and the -Homebrew formula pins CLI archives and checksums. +Homebrew formula pins Amber and Minecart source archives and checksums. ## 1. Framework prerelease @@ -13,11 +13,12 @@ Homebrew formula pins CLI archives and checksums. ## 2. CLI release -1. Confirm `shard.yml` and `AmberCLI::VERSION` both say `2.0.6`. +1. Confirm `shard.yml` and `AmberCLI::VERSION` both say `2.0.7`. 2. Generate a web app and verify it pins the framework prerelease. -3. Verify the generated app pins the reviewed Asset Pipeline revision and has - no placeholder dependency revisions. -4. On macOS, x86-64 Linux, and ARM64 Linux, install shards; run +3. Verify the generated app pins the reviewed Asset Pipeline release exactly, + has a Minecart Git tree checksum for every dependency, and has assistant + files. +4. On macOS, x86-64 Linux, and ARM64 Linux, run `minecart install --frozen`; run `amber assets build` and `amber assets check`; run app specs; build and start the app; then follow the manifest-rendered CSS, JavaScript, SVG, and favicon URLs from `/`. @@ -25,16 +26,19 @@ Homebrew formula pins CLI archives and checksums. caching, `nosniff`, and gzip negotiation. Confirm CSS rewrites its authored image URL and that `public/assets/` contains no source-owned raw entry point. 6. On macOS, reject any binary linked to `openssl@1.1`. -7. Confirm the Windows x86-64 generated app builds in CI. This is a compatibility - check, not a beta release gate. -8. Tag `v2.0.6`, publish the release, and wait for all archives and checksum +7. Confirm the Windows x86-64 generated app passes its full CI smoke, including + launch and asset probes. This is a generated-app release gate, with no + Windows archive yet. +8. Tag `v2.0.7`, publish the release, and wait for all archives and checksum files to upload. ## 3. Homebrew 1. Update the underscore tap, `amberframework/homebrew-amber_cli`. -2. Verify both archive checksums in the formula. -3. Run `brew install amberframework/amber_cli/amber_cli` on macOS and Linux. +2. Verify both source archive checksums in the formula. +3. Run `brew trust --tap amberframework/amber_cli`, pin the reviewed tap commit, + then run `HOMEBREW_NO_AUTO_UPDATE=1 brew install + amberframework/amber_cli/amber_cli` on macOS and Linux. 4. Repeat the complete generated-web-app smoke test from the installed formula. ## 4. Documentation @@ -42,11 +46,11 @@ Homebrew formula pins CLI archives and checksums. 1. Publish framework, CLI, and website guides with the same command names, versions, platform matrix, and preview labels. 2. Check that no onboarding page uses a hyphenated tap or formula name, a - separate tap step, a personal Amber fork, a moving `v2-dev` dependency, or + tap without trust and a commit pin, a personal Amber fork, a moving `v2-dev` dependency, or Slang for a new V2 app. 3. Verify all release/download links and commands from a clean shell. Do not mark the beta complete if macOS, x86-64 Linux, or ARM64 Linux cannot install the CLI and complete the generated web-app and asset smoke. Record a -Windows CI failure as a known compatibility issue; Windows does not gate this -beta. +Windows CI failure as a failed generated-app release gate; Windows packaging +remains a separate open item. diff --git a/scripts/build_release.sh b/scripts/build_release.sh index da6581e..b59c97e 100755 --- a/scripts/build_release.sh +++ b/scripts/build_release.sh @@ -5,9 +5,16 @@ set -euo pipefail -VERSION=${1:-"dev"} +VERSION=${1:?"usage: scripts/build_release.sh X.Y.Z"} OUTPUT_DIR="dist" +test "$VERSION" = "$(awk '/^version:/ { print $2; exit }' shard.yml)" +test -z "$(git status --porcelain)" || { echo "Commit the reviewed source before packaging it" >&2; exit 1; } +command -v crystal-alpha >/dev/null 2>&1 || { echo "crystal-alpha is required" >&2; exit 1; } +command -v minecart >/dev/null 2>&1 || { echo "minecart is required" >&2; exit 1; } +test -s shard.lock +export CRYSTAL_CACHE_DIR="$PWD/.crystal-cache" + echo "🔨 Building Amber CLI v${VERSION}" # Clean previous builds @@ -21,8 +28,8 @@ ARCH=$(uname -m) case "${OS}" in "darwin") TARGET="darwin-arm64" - BUILD_CLI="crystal build src/amber_cli.cr -o amber --release" - BUILD_LSP="crystal build src/amber_lsp.cr -o amber-lsp --release" + BUILD_CLI="crystal-alpha build src/amber_cli.cr -o amber --release" + BUILD_LSP="crystal-alpha build src/amber_lsp.cr -o amber-lsp --release" CHECKSUM_CMD="shasum -a 256" if [ "${ARCH}" != "arm64" ]; then echo "⚠️ Warning: Building for ARM64 on ${ARCH} architecture" @@ -38,8 +45,8 @@ case "${OS}" in exit 1 ;; esac - BUILD_CLI="crystal build src/amber_cli.cr -o amber --release --static" - BUILD_LSP="crystal build src/amber_lsp.cr -o amber-lsp --release --static" + BUILD_CLI="crystal-alpha build src/amber_cli.cr -o amber --release --static" + BUILD_LSP="crystal-alpha build src/amber_lsp.cr -o amber-lsp --release --static" CHECKSUM_CMD="sha256sum" ;; *) @@ -52,11 +59,7 @@ echo "🎯 Building for target: ${TARGET}" # Install dependencies echo "📦 Installing dependencies..." -if [ -f shard.lock ]; then - shards install --production -else - shards install -fi +minecart install --production # Build binaries echo "🔨 Compiling amber CLI..." @@ -75,14 +78,17 @@ test -x amber-lsp # Create archive echo "📦 Creating archive..." tar -czf "${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" amber amber-lsp +git archive --format=tar --prefix="amber_cli-${VERSION}/" HEAD | gzip -n -9 > "${OUTPUT_DIR}/amber_cli-source-${VERSION}.tar.gz" # Calculate checksum echo "🔢 Calculating checksum..." cd "${OUTPUT_DIR}" if command -v sha256sum >/dev/null 2>&1; then sha256sum "amber_cli-${TARGET}.tar.gz" > "amber_cli-${TARGET}.tar.gz.sha256" + sha256sum "amber_cli-source-${VERSION}.tar.gz" > "amber_cli-source-${VERSION}.tar.gz.sha256" else ${CHECKSUM_CMD} "amber_cli-${TARGET}.tar.gz" > "amber_cli-${TARGET}.tar.gz.sha256" + ${CHECKSUM_CMD} "amber_cli-source-${VERSION}.tar.gz" > "amber_cli-source-${VERSION}.tar.gz.sha256" fi SHA256=$(cut -d' ' -f1 < "amber_cli-${TARGET}.tar.gz.sha256") @@ -90,6 +96,7 @@ echo "" echo "🎉 Build complete!" echo "📁 Output: ${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" echo "🔑 SHA256: ${SHA256}" +echo "🔑 Source SHA256: $(cut -d' ' -f1 < "amber_cli-source-${VERSION}.tar.gz.sha256")" echo "" echo "To test the archive:" echo " tar -xzf ${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" diff --git a/scripts/check_beta_contract.sh b/scripts/check_beta_contract.sh index f6aa489..1f0d033 100755 --- a/scripts/check_beta_contract.sh +++ b/scripts/check_beta_contract.sh @@ -3,7 +3,7 @@ set -euo pipefail shard_version="$(awk '/^version:/ { print $2; exit }' shard.yml)" cli_version="$(sed -n 's/.*VERSION = "\([^"]*\)".*/\1/p' src/amber_cli.cr | head -1)" -test "$shard_version" = "2.0.6" +test "$shard_version" = "2.0.7" test "$cli_version" = "$shard_version" grep -F 'github: amberframework/amber' src/amber_cli/commands/new.cr @@ -13,9 +13,9 @@ grep -F 'model: grant' src/amber_cli/commands/new.cr grep -F 'database: #{database}' src/amber_cli/commands/new.cr grep -F 'github: crimson-knight/grant' src/amber_cli/commands/new.cr grep -F 'github: amberframework/asset_pipeline' src/amber_cli/commands/new.cr -grep -F 'version: ~> 0.37.0' src/amber_cli/commands/new.cr +grep -F 'version: 0.37.0' src/amber_cli/commands/new.cr grep -F 'github: amberframework/asset_pipeline' src/amber_cli/templates/app/shard.yml.ecr -grep -F 'version: ~> 0.37.0' src/amber_cli/templates/app/shard.yml.ecr +grep -F 'version: 0.37.0' src/amber_cli/templates/app/shard.yml.ecr grep -F 'github: amberframework/amber' src/amber_cli/generators/native_app.cr grep -F 'version: 2.0.0-beta.5' src/amber_cli/generators/native_app.cr grep -F 'version: ~> 0.37.0' src/amber_cli/generators/native_app.cr @@ -63,7 +63,8 @@ grep -F 'javascript_importmap_tag({"app" => "javascript/app.js"}' src/amber_cli/ grep -F 'favicon_tag("images/favicon.svg")' src/amber_cli/templates/app/src/views/layouts/application.ecr.ecr grep -F 'image_tag("images/amber-crystal.svg"' src/amber_cli/templates/app/src/views/home/index.ecr.ecr grep -F '/public/assets/' src/amber_cli/templates/app/.gitignore.ecr -grep -F 'brew install amberframework/amber_cli/amber_cli' README.md +grep -F 'brew trust --tap amberframework/amber_cli' README.md +grep -F 'HOMEBREW_NO_AUTO_UPDATE=1 brew install amberframework/amber_cli/amber_cli' README.md files=( README.md @@ -75,7 +76,7 @@ files=( docs/*.md ) -if grep -Ein 'amberframework/amber-cli|brew tap amberframework/amber_cli|brew install amber-cli|brew install amber_cli|docs\.amberframework\.org' "${files[@]}" src/amber_cli/*.cr src/amber_cli/commands/*.cr; then +if grep -Ein 'amberframework/amber-cli|brew install amber-cli|brew install amber_cli|docs\.amberframework\.org' "${files[@]}" src/amber_cli/*.cr src/amber_cli/commands/*.cr; then echo "Amber CLI docs contain an untrusted or incomplete Homebrew install path" >&2 exit 1 fi diff --git a/scripts/generate_docs.sh b/scripts/generate_docs.sh index 6d96fd1..e681d49 100755 --- a/scripts/generate_docs.sh +++ b/scripts/generate_docs.sh @@ -22,7 +22,7 @@ if [ ! -f "shard.yml" ]; then fi echo "📦 Installing dependencies..." -shards install +minecart install --frozen echo "📖 Generating documentation..." @@ -43,4 +43,4 @@ echo "✅ Documentation generated successfully in ./docs" echo "" echo "🎉 Done! Documentation is ready." echo " Local files: ./docs/index.html" -echo " Live site: https://amberframework.github.io/amber_cli/" \ No newline at end of file +echo " Live site: https://amberframework.github.io/amber_cli/" diff --git a/scripts/install_pinned_minecart.ps1 b/scripts/install_pinned_minecart.ps1 new file mode 100644 index 0000000..894eb1b --- /dev/null +++ b/scripts/install_pinned_minecart.ps1 @@ -0,0 +1,34 @@ +$ErrorActionPreference = "Stop" + +$minecartCommit = "4fea47ce6561ea0612cf1e0cf6955d782ef27929" +$minecartTree = "e0a0214c14d71f1031d83b760d426b09509b5d9d" +$installRoot = Join-Path $env:RUNNER_TEMP "amber-pinned-minecart" + +New-Item -ItemType Directory -Force $installRoot | Out-Null +& git -C $installRoot init -q +& git -C $installRoot remote add origin https://github.com/crimson-knight/shards.git +& git -C $installRoot fetch --depth=1 origin $minecartCommit +if ($LASTEXITCODE -ne 0) { throw "Could not fetch the pinned Minecart commit" } +& git -C $installRoot checkout -q --detach FETCH_HEAD +if ($LASTEXITCODE -ne 0) { throw "Could not check out the pinned Minecart commit" } + +$actualCommit = (& git -C $installRoot rev-parse HEAD).Trim() +$actualTree = (& git -C $installRoot rev-parse 'HEAD^{tree}').Trim() +if ($actualCommit -ne $minecartCommit -or $actualTree -ne $minecartTree) { + throw "Minecart commit or tree hash did not match the pin" +} + +Push-Location $installRoot +try { + New-Item -ItemType Directory -Force bin | Out-Null + & crystal build src/shards.cr -o bin/minecart.exe + if ($LASTEXITCODE -ne 0) { throw "Minecart build failed" } + $minecartVersion = (& ./bin/minecart.exe --version).Trim() + if ($minecartVersion -notlike "Minecart 2025.11.25.7*") { + throw "Built Minecart version did not match the pin: $minecartVersion" + } +} finally { + Pop-Location +} + +"$(Join-Path $installRoot 'bin')" >> $env:GITHUB_PATH diff --git a/scripts/install_pinned_minecart.sh b/scripts/install_pinned_minecart.sh new file mode 100755 index 0000000..eabcefb --- /dev/null +++ b/scripts/install_pinned_minecart.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +set -euo pipefail + +# The source commit is verified by its Git tree before it can run the web smoke. +minecart_commit=4fea47ce6561ea0612cf1e0cf6955d782ef27929 +minecart_tree=e0a0214c14d71f1031d83b760d426b09509b5d9d +install_root="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/amber-pinned-minecart" + +mkdir -p "$install_root" +git -C "$install_root" init -q +git -C "$install_root" remote add origin https://github.com/crimson-knight/shards.git +git -C "$install_root" fetch --depth=1 origin "$minecart_commit" +git -C "$install_root" checkout -q --detach FETCH_HEAD +test "$(git -C "$install_root" rev-parse HEAD)" = "$minecart_commit" +test "$(git -C "$install_root" rev-parse 'HEAD^{tree}')" = "$minecart_tree" + +make -C "$install_root" bin/minecart CRYSTAL="${CRYSTAL:-crystal}" +test -x "$install_root/bin/minecart" +"$install_root/bin/minecart" --version | grep -F "Minecart 2025.11.25.7" + +if [[ -n "${GITHUB_PATH:-}" ]]; then + echo "$install_root/bin" >> "$GITHUB_PATH" +else + echo "Add $install_root/bin to PATH before running the web smoke." +fi diff --git a/scripts/smoke_generated_web.ps1 b/scripts/smoke_generated_web.ps1 index 40f2192..d5a0340 100644 --- a/scripts/smoke_generated_web.ps1 +++ b/scripts/smoke_generated_web.ps1 @@ -24,6 +24,7 @@ function Invoke-Checked { } $cliPath = (Resolve-Path $Cli).Path +$null = Get-Command minecart -ErrorAction Stop $smokeRoot = Join-Path $env:RUNNER_TEMP "amber-v2-windows-smoke" $appPath = Join-Path $smokeRoot "amber_beta_smoke" @@ -32,7 +33,7 @@ if (Test-Path $smokeRoot) { } New-Item -ItemType Directory -Force $smokeRoot | Out-Null -Invoke-Checked -Command $cliPath -Arguments @("new", $appPath, "--type", "web", "--no-deps") +Invoke-Checked -Command $cliPath -Arguments @("new", $appPath, "--type", "web", "-y") $shardPath = Join-Path $appPath "shard.yml" $amberConfigPath = Join-Path $appPath ".amber.yml" @@ -54,9 +55,15 @@ if (-not $manifest.Contains("github: crystal-lang/crystal-sqlite3")) { if (-not $manifest.Contains("github: amberframework/asset_pipeline")) { throw "Generated shard.yml does not include Asset Pipeline" } -if (-not $manifest.Contains("version: ~> 0.37.0")) { +if (-not $manifest.Contains("version: 0.37.0")) { throw "Generated shard.yml does not pin the supported Asset Pipeline release" } +if (-not (Test-Path (Join-Path $appPath ".claude/CLAUDE.md"))) { + throw "Minecart did not install assistant files" +} +if (-not ([System.IO.File]::ReadAllText((Join-Path $appPath ".minecart-policy.yml")).Contains("require_exact: true"))) { + throw "Generated app does not require exact root dependency pins" +} if (-not $amberConfig.Contains("database: sqlite") -or -not $amberConfig.Contains("model: grant")) { throw "Generated .amber.yml does not select SQLite and Grant" } @@ -84,7 +91,17 @@ if ($FrameworkCommit) { Push-Location $appPath try { Invoke-Checked -Command $cliPath -Arguments @("assets", "check") - Invoke-Checked -Command "shards" -Arguments @("install") + if ($FrameworkCommit) { + Invoke-Checked -Command "minecart" -Arguments @("install", "--strict-pinning") + } else { + Invoke-Checked -Command "minecart" -Arguments @("install", "--frozen") + } + $lock = [System.IO.File]::ReadAllText((Join-Path $appPath "shard.lock")) + $shardCount = [regex]::Matches($lock, '(?m)^ [A-Za-z0-9_-]+:$').Count + $checksumCount = [regex]::Matches($lock, '(?m)^ checksum: git-tree:[0-9a-f]{40}$').Count + if ($shardCount -eq 0 -or $checksumCount -ne $shardCount) { + throw "Minecart lock is missing a git-tree checksum for a dependency" + } Invoke-Checked -Command $cliPath -Arguments @("assets", "build") Invoke-Checked -Command $cliPath -Arguments @("assets", "check") diff --git a/scripts/smoke_generated_web.sh b/scripts/smoke_generated_web.sh index 903077a..ead385a 100755 --- a/scripts/smoke_generated_web.sh +++ b/scripts/smoke_generated_web.sh @@ -9,17 +9,13 @@ fi cli_path="$1" framework_commit="${2:-}" framework_repository="${3:-amberframework/amber}" -if command -v shards-alpha >/dev/null 2>&1; then - shards_command="shards-alpha" -else - shards_command="shards" -fi +command -v minecart >/dev/null 2>&1 || { echo "Minecart is required for the generated web smoke" >&2; exit 1; } if command -v crystal-alpha >/dev/null 2>&1; then crystal_command="crystal-alpha" else crystal_command="crystal" fi -echo "Using dependency installer: $shards_command" +echo "Using dependency installer: $(command -v minecart)" script_dir="$(cd "$(dirname "$0")" && pwd)" if [[ "$cli_path" != /* ]]; then cli_path="$(cd "$(dirname "$cli_path")" && pwd)/$(basename "$cli_path")" @@ -42,10 +38,9 @@ trap cleanup EXIT "$cli_path" new "$app_path" --type web test -s "$app_path/shard.lock" -grep -F 'shards install' "$app_path/README.md" +grep -F 'minecart install --frozen' "$app_path/README.md" grep -F 'crystal spec' "$app_path/README.md" -grep -F '`shards-alpha install` writes a checksum-verified `shard.lock` when `shards-alpha` is installed; commit the lock with the application.' "$app_path/README.md" -test "$(grep -cF '`shards-alpha install` writes a checksum-verified `shard.lock` when `shards-alpha` is installed; commit the lock with the application.' "$app_path/README.md")" -eq 1 +grep -F 'git-tree:' "$app_path/README.md" ! grep -F 'crystal-alpha' "$app_path/.amber.yml" grep -F 'AMBER_ENV=test crystal spec' "$script_dir/../src/amber_cli/templates/app/.amber.yml.ecr" ! grep -F 'crystal-alpha' "$script_dir/../src/amber_cli/templates/app/.amber.yml.ecr" @@ -54,7 +49,7 @@ grep -F "github: amberframework/amber" "$app_path/shard.yml" grep -F "version: 2.0.0-beta.5" "$app_path/shard.yml" grep -F "github: crimson-knight/grant" "$app_path/shard.yml" grep -F "github: amberframework/asset_pipeline" "$app_path/shard.yml" -grep -F "version: ~> 0.37.0" "$app_path/shard.yml" +grep -F "version: 0.37.0" "$app_path/shard.yml" grep -F "commit: da1e06161148f156dbce262a4a4efcb39cba5ba4" "$app_path/shard.yml" grep -F "github: crystal-lang/crystal-sqlite3" "$app_path/shard.yml" grep -F "template: ecr" "$app_path/.amber.yml" @@ -79,24 +74,22 @@ fi cd "$app_path" test -s shard.lock +test -s .claude/CLAUDE.md +grep -F 'require_exact: true' .minecart-policy.yml ! grep -F "shard.lock" .gitignore "$cli_path" assets check -env GIT_CONFIG_COUNT=1 \ - GIT_CONFIG_KEY_0=core.hooksPath \ - GIT_CONFIG_VALUE_0=/dev/null \ - "$shards_command" install +if [[ -n "$framework_commit" ]]; then + minecart install --strict-pinning +else + minecart install --frozen +fi test -s shard.lock grep -F "version: 0.23.4+git.commit.da1e06161148f156dbce262a4a4efcb39cba5ba4" shard.lock grep -F "version: 0.14.0" shard.lock -if [[ "$shards_command" == "shards-alpha" ]]; then - grep -F "checksum: sha256:ad693a4f04294abbb0c827a34da141f1e9cd0a96293fc4d7039d55e1b8eb3005" shard.lock - grep -F "checksum: sha256:4a4dea15c27b985cd9d8c5c545024daafab347b56986252fd24fcc3f18e5c024" shard.lock - shard_count="$(grep -cE '^ [[:alnum:]_-]+:$' shard.lock)" - checksum_count="$(grep -cE '^ checksum: sha256:[0-9a-f]{64}$' shard.lock)" - test "$checksum_count" -eq "$shard_count" -else - echo "shards-alpha unavailable; checksum lock assertions skipped" >&2 -fi +shard_count="$(grep -cE '^ [[:alnum:]_-]+:$' shard.lock)" +checksum_count="$(grep -cE '^ checksum: git-tree:[0-9a-f]{40}$' shard.lock)" +test "$shard_count" -gt 0 +test "$checksum_count" -eq "$shard_count" "$cli_path" assets build "$cli_path" assets check "$crystal_command" spec diff --git a/shard.lock b/shard.lock index febdfca..121a2f4 100644 --- a/shard.lock +++ b/shard.lock @@ -3,40 +3,50 @@ shards: ameba: git: https://github.com/crystal-ameba/ameba.git version: 1.7.0-dev+git.commit.cdd58b34b0d8a9c785d67183a7a8f07541549e55 + checksum: git-tree:5fe9fbff073fc9a41923f19d8a43b9b35a528ac8 asset_pipeline: git: https://github.com/amberframework/asset_pipeline.git version: 0.37.0 + checksum: git-tree:47c884c51cc63e62d04056bc160ab6c329e1946c backtracer: git: https://github.com/sija/backtracer.cr.git version: 1.2.4 + checksum: git-tree:54afe1195b6229aae7ee38849f857a7d8dc5788f compiled_license: git: https://github.com/elorest/compiled_license.git version: 1.2.2 + checksum: git-tree:3e2663f5a593c9a83d05cd9052e3933f45bb2c18 db: git: https://github.com/crystal-lang/crystal-db.git version: 0.13.1 + checksum: git-tree:77be88f3f39ff1d7cbba1022062a69d09b763de4 exception_page: git: https://github.com/crystal-loot/exception_page.git version: 0.2.2 + checksum: git-tree:460d68c049fdca6ee6160fdfdf7d58b0c60ae82e micrate: git: https://github.com/amberframework/micrate.git version: 0.16.0-beta.1+git.commit.56afd5607cc591776980f3d3762495c037b98d98 + checksum: git-tree:b8fe378a82510aaeb61b6f4306099025272ddf52 mysql: git: https://github.com/crystal-lang/crystal-mysql.git version: 0.16.0 + checksum: git-tree:7c87e8d1a35b97c3b699df61eab28099a630a856 pg: git: https://github.com/will/crystal-pg.git version: 0.29.0 + checksum: git-tree:9537ebbb1a6b936bfb9cc87307d3b661f70ab518 sqlite3: git: https://github.com/crystal-lang/crystal-sqlite3.git version: 0.21.0 + checksum: git-tree:0724768c38a3e14b9a6e9161515c4e293a7f1e0c diff --git a/shard.yml b/shard.yml index d6f323b..e9e9852 100644 --- a/shard.yml +++ b/shard.yml @@ -1,5 +1,5 @@ name: amber_cli -version: 2.0.6 +version: 2.0.7 authors: - crimson-knight @@ -19,7 +19,7 @@ dependencies: # Stable static-asset compiler used by the CLI and generated applications. asset_pipeline: github: amberframework/asset_pipeline - version: ~> 0.37.0 + version: 0.37.0 micrate: github: amberframework/micrate @@ -27,24 +27,24 @@ dependencies: pg: github: will/crystal-pg - version: ~> 0.29.0 + version: 0.29.0 mysql: github: crystal-lang/crystal-mysql - version: ~> 0.16.0 + version: 0.16.0 sqlite3: github: crystal-lang/crystal-sqlite3 - version: ~> 0.21.0 + version: 0.21.0 exception_page: github: crystal-loot/exception_page - version: ~> 0.2.1 + version: 0.2.2 # CLI tool item, specifically for when distributing a compiled binary (like the CLI tool will be) compiled_license: github: elorest/compiled_license - version: ~> 1.2.2 + version: 1.2.2 development_dependencies: ameba: diff --git a/spec/commands/new_command_spec.cr b/spec/commands/new_command_spec.cr index 50da65f..3044fba 100644 --- a/spec/commands/new_command_spec.cr +++ b/spec/commands/new_command_spec.cr @@ -2,26 +2,26 @@ require "../amber_cli_spec" require "../../src/amber_cli/commands/new" describe AmberCLI::Commands::NewCommand do - describe ".find_shards_executable" do - it "prefers shards-alpha when it is available" do - executable = AmberCLI::Commands::NewCommand.find_shards_executable( - ->(command : String) { command == "shards-alpha" ? "/tools/shards-alpha" : "/tools/shards" } - ) - - executable.should eq("/tools/shards-alpha") - end - - it "falls back to shards when shards-alpha is unavailable" do + describe ".find_minecart_executable" do + it "selects Minecart for checksum-verified installs" do lookup_calls = [] of String - executable = AmberCLI::Commands::NewCommand.find_shards_executable( + executable = AmberCLI::Commands::NewCommand.find_minecart_executable( ->(command : String) do lookup_calls << command - command == "shards" ? "/tools/shards" : nil + command == "minecart" ? "/tools/minecart" : "/tools/other" end ) - executable.should eq("/tools/shards") - lookup_calls.should eq(["shards-alpha", "shards"]) + executable.should eq("/tools/minecart") + lookup_calls.should eq(["minecart"]) + end + + it "returns nil rather than selecting an unverified installer" do + executable = AmberCLI::Commands::NewCommand.find_minecart_executable( + ->(command : String) { nil } + ) + + executable.should be_nil end end @@ -115,15 +115,19 @@ describe AmberCLI::Commands::NewCommand do shard.should contain("commit: da1e06161148f156dbce262a4a4efcb39cba5ba4") shard.should contain("asset_pipeline:") shard.should contain("github: amberframework/asset_pipeline") + shard.should contain("version: 0.37.0") + shard.should_not contain("version: ~> 0.37.0") shard.should contain("github: crystal-lang/crystal-sqlite3") shard.should_not contain("slang") File.exists?(File.join(destination, "shard.lock")).should be_false + policy = File.read(File.join(destination, ".minecart-policy.yml")) + policy.should contain("require_exact: true") readme = File.read(File.join(destination, "README.md")) - readme.should contain("shards install") + readme.should contain("minecart install --frozen") readme.should contain("crystal spec") - readme.should contain("writes a checksum-verified `shard.lock`") - readme.scan(/shards-alpha install/).size.should eq(1) + readme.should contain("git-tree:") + readme.should_not contain("shards install") amber_config_template = File.read(File.expand_path("../../src/amber_cli/templates/app/.amber.yml.ecr", __DIR__)) amber_config_template.should contain("AMBER_ENV=test crystal spec") diff --git a/src/amber_cli.cr b/src/amber_cli.cr index b886e52..bcc8203 100644 --- a/src/amber_cli.cr +++ b/src/amber_cli.cr @@ -40,7 +40,7 @@ end Log.builder.bind "*", :info, backend module AmberCLI - VERSION = "2.0.6" + VERSION = "2.0.7" def self.run(args = ARGV) if args.empty? diff --git a/src/amber_cli/commands/new.cr b/src/amber_cli/commands/new.cr index e7e3a2e..8342dac 100644 --- a/src/amber_cli/commands/new.cr +++ b/src/amber_cli/commands/new.cr @@ -44,8 +44,8 @@ module AmberCLI::Commands getter name : String = "" # :nodoc: - def self.find_shards_executable(path_lookup : Proc(String, String?)) : String? - path_lookup.call("shards-alpha") || path_lookup.call("shards") + def self.find_minecart_executable(path_lookup : Proc(String, String?)) : String? + path_lookup.call("minecart") end def help_description : String @@ -192,7 +192,11 @@ module AmberCLI::Commands puts "" info "To get started:" info " cd #{name}" unless name == "." - info " shards install" if no_deps + if no_deps + info " minecart install --strict-pinning" + info " minecart assistant init" + end + info " minecart install --frozen # For later installs" info " crystal spec" info " amber generate scaffold Pet name:string:required species:string:required" info " amber database migrate" @@ -201,11 +205,32 @@ module AmberCLI::Commands end private def install_dependencies(path : String) - info "Installing dependencies..." - shards_executable = self.class.find_shards_executable(->(command : String) { Process.find_executable(command) }) + minecart = self.class.find_minecart_executable(->(command : String) { Process.find_executable(command) }) + unless minecart + error "Minecart is required to create a checksum-verified Amber web app." + info "Install Minecart with Amber, then run 'minecart install --strict-pinning' in #{path}." + exit!(error: true) + end + + info "Installing dependencies with Minecart..." + run_minecart(minecart, ["install", "--strict-pinning"], path) + verify_minecart_lock(path) + + info "Installing Minecart assistant files..." + run_minecart(minecart, ["assistant", "init"], path) + unless File.exists?(File.join(path, ".claude", "CLAUDE.md")) + error "Minecart assistant setup did not create .claude/CLAUDE.md; the project files were kept." + exit!(error: true) + end + rescue ex : File::NotFoundError + error "Minecart disappeared during project setup; the project files were kept." + exit!(error: true) + end + + private def run_minecart(executable : String, arguments : Array(String), path : String) status = Process.run( - shards_executable || "shards", - ["install"], + executable, + arguments, chdir: path, input: Process::Redirect::Inherit, output: Process::Redirect::Inherit, @@ -213,15 +238,26 @@ module AmberCLI::Commands ) return if status.success? - warning "Dependency installation failed; the project files were kept." - warning "Run 'shards install' in #{path} after resolving the error." - exit!(error: true) - rescue ex : File::NotFoundError - warning "The 'shards' executable was not found; the project files were kept." - warning "Install Crystal and run 'shards install' in #{path}." + error "Minecart #{arguments.join(" ")} failed; the project files were kept." exit!(error: true) end + private def verify_minecart_lock(path : String) + lock_path = File.join(path, "shard.lock") + unless File.exists?(lock_path) + error "Minecart completed without shard.lock; the project files were kept." + exit!(error: true) + end + + lock = File.read(lock_path) + dependency_count = lock.lines.count { |line| /^ [^\s:]+:$/.matches?(line.chomp) } + checksum_count = lock.lines.count { |line| /^ checksum: git-tree:[0-9a-f]{40}$/.matches?(line.chomp) } + if dependency_count == 0 || checksum_count != dependency_count + error "Minecart did not lock every dependency with a git-tree checksum; the project files were kept." + exit!(error: true) + end + end + private def create_project_structure(path : String, name : String) # Create V2 directory structure dirs = [ @@ -252,6 +288,7 @@ module AmberCLI::Commands create_amber_yml(path, name) create_readme(path, name) create_gitignore(path) + create_minecart_policy(path) create_main_file(path, name) create_config_files(path, name) create_assets_config(path) @@ -296,7 +333,7 @@ dependencies: commit: da1e06161148f156dbce262a4a4efcb39cba5ba4 asset_pipeline: github: amberframework/asset_pipeline - version: ~> 0.37.0 + version: 0.37.0 #{database_shard_dependency} SHARD @@ -327,7 +364,7 @@ An ECR web application generated by Amber CLI for Amber `2.0.0-beta.5`. ## Run it ```bash -shards install +minecart install --frozen amber assets build amber assets check crystal spec @@ -336,7 +373,11 @@ amber database migrate amber watch ``` -`shards-alpha install` writes a checksum-verified `shard.lock` when `shards-alpha` is installed; commit the lock with the application. +`amber new` uses Minecart to write a `shard.lock` with `git-tree:` checksums and +install `.claude/` assistant files. Commit the lock and `.minecart-policy.yml`. +For later installs, use `minecart install --frozen`. If the project was created +with `--no-deps`, first run `minecart install --strict-pinning` and +`minecart assistant init`, then commit the lock. Open for the starter page or after generating the example scaffold. @@ -407,6 +448,17 @@ GITIGNORE write_text(File.join(path, ".gitignore"), gitignore_content) end + private def create_minecart_policy(path : String) + policy_content = <<-POLICY +version: 1 +rules: + dependencies: + require_exact: true +POLICY + + write_text(File.join(path, ".minecart-policy.yml"), policy_content) + end + private def create_main_file(path : String, name : String) main_content = <<-MAIN require "../config/*" diff --git a/src/amber_cli/templates/app/shard.yml.ecr b/src/amber_cli/templates/app/shard.yml.ecr index 3b9e231..e717355 100644 --- a/src/amber_cli/templates/app/shard.yml.ecr +++ b/src/amber_cli/templates/app/shard.yml.ecr @@ -21,7 +21,7 @@ dependencies: commit: da1e06161148f156dbce262a4a4efcb39cba5ba4 asset_pipeline: github: amberframework/asset_pipeline - version: ~> 0.37.0 + version: 0.37.0 <% case @database when "pg" -%> pg: From 0331e234f3428ec274eed72c90fb0b0ddecc6156 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Sun, 4 Oct 2026 14:22:20 -0400 Subject: [PATCH 07/69] fix(onboarding): keep Minecart assistant setup compact --- .github/workflows/build.yml | 2 +- .github/workflows/ci.yml | 6 +++--- .github/workflows/docs.yml | 2 +- .github/workflows/platform-compile.yml | 4 ++-- .github/workflows/release.yml | 2 +- README.md | 6 +++--- docs/BETA_WEB_APP.md | 2 +- docs/RELEASE_CHECKLIST.md | 2 +- scripts/build_release.sh | 2 +- scripts/generate_docs.sh | 2 +- scripts/smoke_generated_web.ps1 | 8 ++++++-- scripts/smoke_generated_web.sh | 6 ++++-- spec/commands/new_command_spec.cr | 1 + src/amber_cli/commands/new.cr | 17 ++++++++++------- src/amber_cli/templates/app/README.md.ecr | 7 +++++-- 15 files changed, 41 insertions(+), 28 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index ab241c9..4074281 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -55,7 +55,7 @@ jobs: run: scripts/install_pinned_minecart.sh - name: Install dependencies - run: minecart install --frozen + run: minecart install --frozen --skip-ai-docs - name: Build LSP binary for integration specs run: | diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cf89f2d..e50a167 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -47,7 +47,7 @@ jobs: run: scripts/install_pinned_minecart.sh - name: Install dependencies - run: minecart install --frozen + run: minecart install --frozen --skip-ai-docs - name: Check Amber V2 beta contract run: scripts/check_beta_contract.sh @@ -127,7 +127,7 @@ jobs: run: scripts/install_pinned_minecart.sh - name: Install dependencies - run: minecart install --frozen + run: minecart install --frozen --skip-ai-docs - name: Test CLI functionality run: | @@ -165,7 +165,7 @@ jobs: run: scripts/install_pinned_minecart.sh - name: Install dependencies - run: minecart install --frozen + run: minecart install --frozen --skip-ai-docs - name: Run integration tests run: | diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 14968a6..9a2f79f 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -43,7 +43,7 @@ jobs: run: scripts/install_pinned_minecart.sh - name: Install dependencies - run: minecart install --frozen + run: minecart install --frozen --skip-ai-docs - name: Generate documentation run: | diff --git a/.github/workflows/platform-compile.yml b/.github/workflows/platform-compile.yml index 229d2f2..c99eb76 100644 --- a/.github/workflows/platform-compile.yml +++ b/.github/workflows/platform-compile.yml @@ -30,7 +30,7 @@ jobs: run: scripts/install_pinned_minecart.sh - name: Install CLI dependencies - run: minecart install --frozen + run: minecart install --frozen --skip-ai-docs - name: Build Amber CLI run: crystal build src/amber_cli.cr --no-debug -o amber @@ -70,7 +70,7 @@ jobs: run: scripts/install_pinned_minecart.ps1 - name: Install CLI dependencies - run: minecart install --frozen + run: minecart install --frozen --skip-ai-docs - name: Build Amber CLI run: crystal build src/amber_cli.cr --no-debug -o amber.exe diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 039548f..d5afe04 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -68,7 +68,7 @@ jobs: run: scripts/install_pinned_minecart.sh - name: Install dependencies - run: minecart install --production + run: minecart install --production --skip-ai-docs - name: Build binaries (Linux) if: startsWith(matrix.target, 'linux-') diff --git a/README.md b/README.md index 7b6f30e..396b3df 100644 --- a/README.md +++ b/README.md @@ -63,7 +63,7 @@ command with `sudo` if `/usr/local/bin` is not writable. ```bash amber new my_app --type web -y cd my_app -minecart install --frozen +minecart install --frozen --skip-ai-docs amber assets check crystal spec crystal build src/my_app.cr -o bin/my_app @@ -74,7 +74,7 @@ amber watch a `shard.lock` with Git tree checksums, `.minecart-policy.yml`, and `.claude/` assistant files. Commit those files. With `--no-deps`, the first Minecart install and assistant setup are deliberately deferred; run `minecart install ---strict-pinning` and `minecart assistant init` before frozen installs. +--strict-pinning --skip-ai-docs` and `minecart assistant init` before frozen installs. `amber watch` recompiles assets before the application whenever an ECR template or a file under `app/assets/` changes. Open . @@ -191,7 +191,7 @@ See the [LSP setup guide](https://github.com/amberframework/amber/blob/v2.0.0-be ## Contributing ```bash -minecart install --frozen +minecart install --frozen --skip-ai-docs crystal-alpha tool format --check src spec crystal-alpha spec ``` diff --git a/docs/BETA_WEB_APP.md b/docs/BETA_WEB_APP.md index 333b226..195b36a 100644 --- a/docs/BETA_WEB_APP.md +++ b/docs/BETA_WEB_APP.md @@ -20,7 +20,7 @@ newer. ```bash amber new amber_beta_smoke --type web -y cd amber_beta_smoke -minecart install --frozen +minecart install --frozen --skip-ai-docs ``` The command uses Minecart to install dependencies, write a checksummed diff --git a/docs/RELEASE_CHECKLIST.md b/docs/RELEASE_CHECKLIST.md index 6c176cc..21af7cf 100644 --- a/docs/RELEASE_CHECKLIST.md +++ b/docs/RELEASE_CHECKLIST.md @@ -18,7 +18,7 @@ Homebrew formula pins Amber and Minecart source archives and checksums. 3. Verify the generated app pins the reviewed Asset Pipeline release exactly, has a Minecart Git tree checksum for every dependency, and has assistant files. -4. On macOS, x86-64 Linux, and ARM64 Linux, run `minecart install --frozen`; run +4. On macOS, x86-64 Linux, and ARM64 Linux, run `minecart install --frozen --skip-ai-docs`; run `amber assets build` and `amber assets check`; run app specs; build and start the app; then follow the manifest-rendered CSS, JavaScript, SVG, and favicon URLs from `/`. diff --git a/scripts/build_release.sh b/scripts/build_release.sh index b59c97e..6a8391c 100755 --- a/scripts/build_release.sh +++ b/scripts/build_release.sh @@ -59,7 +59,7 @@ echo "🎯 Building for target: ${TARGET}" # Install dependencies echo "📦 Installing dependencies..." -minecart install --production +minecart install --production --skip-ai-docs # Build binaries echo "🔨 Compiling amber CLI..." diff --git a/scripts/generate_docs.sh b/scripts/generate_docs.sh index e681d49..547d131 100755 --- a/scripts/generate_docs.sh +++ b/scripts/generate_docs.sh @@ -22,7 +22,7 @@ if [ ! -f "shard.yml" ]; then fi echo "📦 Installing dependencies..." -minecart install --frozen +minecart install --frozen --skip-ai-docs echo "📖 Generating documentation..." diff --git a/scripts/smoke_generated_web.ps1 b/scripts/smoke_generated_web.ps1 index d5a0340..f6edcb3 100644 --- a/scripts/smoke_generated_web.ps1 +++ b/scripts/smoke_generated_web.ps1 @@ -61,6 +61,10 @@ if (-not $manifest.Contains("version: 0.37.0")) { if (-not (Test-Path (Join-Path $appPath ".claude/CLAUDE.md"))) { throw "Minecart did not install assistant files" } +$assistantBytes = (Get-ChildItem -Path (Join-Path $appPath ".claude") -File -Recurse | Measure-Object -Property Length -Sum).Sum +if ($assistantBytes -ge 10MB) { + throw "Minecart copied optional dependency AI docs into the beginner app" +} if (-not ([System.IO.File]::ReadAllText((Join-Path $appPath ".minecart-policy.yml")).Contains("require_exact: true"))) { throw "Generated app does not require exact root dependency pins" } @@ -92,9 +96,9 @@ Push-Location $appPath try { Invoke-Checked -Command $cliPath -Arguments @("assets", "check") if ($FrameworkCommit) { - Invoke-Checked -Command "minecart" -Arguments @("install", "--strict-pinning") + Invoke-Checked -Command "minecart" -Arguments @("install", "--strict-pinning", "--skip-ai-docs") } else { - Invoke-Checked -Command "minecart" -Arguments @("install", "--frozen") + Invoke-Checked -Command "minecart" -Arguments @("install", "--frozen", "--skip-ai-docs") } $lock = [System.IO.File]::ReadAllText((Join-Path $appPath "shard.lock")) $shardCount = [regex]::Matches($lock, '(?m)^ [A-Za-z0-9_-]+:$').Count diff --git a/scripts/smoke_generated_web.sh b/scripts/smoke_generated_web.sh index ead385a..3f2d0a3 100755 --- a/scripts/smoke_generated_web.sh +++ b/scripts/smoke_generated_web.sh @@ -75,13 +75,15 @@ fi cd "$app_path" test -s shard.lock test -s .claude/CLAUDE.md +assistant_kib="$(du -sk .claude | awk '{print $1}')" +test "$assistant_kib" -lt 10240 grep -F 'require_exact: true' .minecart-policy.yml ! grep -F "shard.lock" .gitignore "$cli_path" assets check if [[ -n "$framework_commit" ]]; then - minecart install --strict-pinning + minecart install --strict-pinning --skip-ai-docs else - minecart install --frozen + minecart install --frozen --skip-ai-docs fi test -s shard.lock grep -F "version: 0.23.4+git.commit.da1e06161148f156dbce262a4a4efcb39cba5ba4" shard.lock diff --git a/spec/commands/new_command_spec.cr b/spec/commands/new_command_spec.cr index 3044fba..129f82a 100644 --- a/spec/commands/new_command_spec.cr +++ b/spec/commands/new_command_spec.cr @@ -125,6 +125,7 @@ describe AmberCLI::Commands::NewCommand do policy.should contain("require_exact: true") readme = File.read(File.join(destination, "README.md")) readme.should contain("minecart install --frozen") + readme.should contain("--skip-ai-docs") readme.should contain("crystal spec") readme.should contain("git-tree:") readme.should_not contain("shards install") diff --git a/src/amber_cli/commands/new.cr b/src/amber_cli/commands/new.cr index 8342dac..7d5dd8f 100644 --- a/src/amber_cli/commands/new.cr +++ b/src/amber_cli/commands/new.cr @@ -193,10 +193,10 @@ module AmberCLI::Commands info "To get started:" info " cd #{name}" unless name == "." if no_deps - info " minecart install --strict-pinning" + info " minecart install --strict-pinning --skip-ai-docs" info " minecart assistant init" end - info " minecart install --frozen # For later installs" + info " minecart install --frozen --skip-ai-docs # For later installs" info " crystal spec" info " amber generate scaffold Pet name:string:required species:string:required" info " amber database migrate" @@ -208,12 +208,12 @@ module AmberCLI::Commands minecart = self.class.find_minecart_executable(->(command : String) { Process.find_executable(command) }) unless minecart error "Minecart is required to create a checksum-verified Amber web app." - info "Install Minecart with Amber, then run 'minecart install --strict-pinning' in #{path}." + info "Install Minecart with Amber, then run 'minecart install --strict-pinning --skip-ai-docs' in #{path}." exit!(error: true) end info "Installing dependencies with Minecart..." - run_minecart(minecart, ["install", "--strict-pinning"], path) + run_minecart(minecart, ["install", "--strict-pinning", "--skip-ai-docs"], path) verify_minecart_lock(path) info "Installing Minecart assistant files..." @@ -364,7 +364,7 @@ An ECR web application generated by Amber CLI for Amber `2.0.0-beta.5`. ## Run it ```bash -minecart install --frozen +minecart install --frozen --skip-ai-docs amber assets build amber assets check crystal spec @@ -375,10 +375,13 @@ amber watch `amber new` uses Minecart to write a `shard.lock` with `git-tree:` checksums and install `.claude/` assistant files. Commit the lock and `.minecart-policy.yml`. -For later installs, use `minecart install --frozen`. If the project was created -with `--no-deps`, first run `minecart install --strict-pinning` and +For later installs, use `minecart install --frozen --skip-ai-docs`. If the project was created +with `--no-deps`, first run `minecart install --strict-pinning --skip-ai-docs` and `minecart assistant init`, then commit the lock. +Dependency AI docs can be large and are skipped by default. Minecart's own +assistant config and skills are still installed. + Open for the starter page or after generating the example scaffold. diff --git a/src/amber_cli/templates/app/README.md.ecr b/src/amber_cli/templates/app/README.md.ecr index fb1f256..d7892bb 100644 --- a/src/amber_cli/templates/app/README.md.ecr +++ b/src/amber_cli/templates/app/README.md.ecr @@ -5,7 +5,7 @@ An ECR web application generated by Amber CLI for Amber `2.0.0-beta.5`. ## Run it ```bash -shards install +minecart install --frozen --skip-ai-docs amber assets build amber assets check crystal spec @@ -14,7 +14,10 @@ amber database migrate amber watch ``` -`shards-alpha install` writes a checksum-verified `shard.lock` when `shards-alpha` is installed; commit the lock with the application. +`amber new` uses Minecart to write a checksum-verified `shard.lock` and install +assistant files. Commit the lock and `.minecart-policy.yml` with the application. +Dependency AI docs are skipped because they can be large; Minecart's own +assistant setup still runs. Open . Authored CSS, JavaScript, images, fonts, and other files live under `app/assets/`. `amber assets build` fingerprints them into the From 33f66da416d21b6dc8f0e11f1f9672bac2111268 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Sun, 4 Oct 2026 14:30:03 -0400 Subject: [PATCH 08/69] docs(release): align source archive instructions with workflow --- .github/workflows/release.yml | 4 ++-- RELEASE_SETUP.md | 17 ++++++++++------- docs/RELEASE_CHECKLIST.md | 15 +++++++-------- 3 files changed, 19 insertions(+), 17 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d5afe04..f4b8c10 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -182,8 +182,8 @@ jobs: artifacts/amber_cli-linux-x86_64/amber_cli-linux-x86_64.tar.gz.sha256 artifacts/amber_cli-linux-arm64/amber_cli-linux-arm64.tar.gz artifacts/amber_cli-linux-arm64/amber_cli-linux-arm64.tar.gz.sha256 - dist/amber_cli-source-2.0.7.tar.gz - dist/amber_cli-source-2.0.7.tar.gz.sha256 + dist/amber_cli-source-*.tar.gz + dist/amber_cli-source-*.tar.gz.sha256 tag_name: ${{ github.event.release.tag_name }} fail_on_unmatched_files: true env: diff --git a/RELEASE_SETUP.md b/RELEASE_SETUP.md index dc76295..9a171c6 100644 --- a/RELEASE_SETUP.md +++ b/RELEASE_SETUP.md @@ -7,8 +7,8 @@ This guide documents the current Amber CLI release path and the checks we expect A successful release means all of the following have been reviewed and run: 1. A published GitHub release in `amberframework/amber_cli` builds macOS and Linux binaries. -2. The workflow uploads binary archives and checksum files. A source archive - from the reviewed commit is uploaded with its recorded SHA-256 for Homebrew. +2. The workflow uploads binary archives and checksum files. It packages a + source archive from the tagged commit and uploads its SHA-256 for Homebrew. 3. The reviewed tap commit contains the Amber and Minecart source archive URLs and SHA-256 values. It is pushed only after both archives are available. 4. Tap install validation passes on supported Homebrew platforms. @@ -103,11 +103,14 @@ Publishing the release triggers the automated flow: 1. build macOS and Linux binaries 2. upload archives and checksums to the release -3. publish the binary archive checksums alongside the archives - -Upload the prepared `amber_cli-source-2.0.7.tar.gz` archive without rebuilding -it. Recompute its SHA-256 and compare it with `Formula/amber_cli.rb` before -pushing the tap commit. Then run the tap smoke on macOS and Linux. +3. package the tagged source and publish its archive and checksum + +Before pushing the tap commit, download the published +`amber_cli-source-2.0.7.tar.gz` archive, verify its sidecar checksum, and compare +its SHA-256 with `Formula/amber_cli.rb`. The CI-generated gzip archive can +have different bytes from a local archive of the same commit, so update the +formula to the published asset hash when they differ. Then run the tap smoke +on macOS and Linux. ## CI Gates To Check diff --git a/docs/RELEASE_CHECKLIST.md b/docs/RELEASE_CHECKLIST.md index 21af7cf..9663286 100644 --- a/docs/RELEASE_CHECKLIST.md +++ b/docs/RELEASE_CHECKLIST.md @@ -3,13 +3,11 @@ Release order matters because generated apps pin a framework version and the Homebrew formula pins Amber and Minecart source archives and checksums. -## 1. Framework prerelease +## 1. Framework prerequisite -1. Confirm `shard.yml`, `src/amber/version.cr`, and the changelog all say - `2.0.0-beta.5`. -2. Run framework specs and formatting on macOS and Linux. -3. Tag the reviewed `v2-dev` commit as `v2.0.0-beta.5`. -4. Publish it as a GitHub prerelease with migration and support-matrix links. +1. Confirm the published `v2.0.0-beta.5` prerelease has its source archive and + checksum and matches the generated app's exact framework pin. +2. Check the framework's migration and support-matrix links. ## 2. CLI release @@ -29,8 +27,9 @@ Homebrew formula pins Amber and Minecart source archives and checksums. 7. Confirm the Windows x86-64 generated app passes its full CI smoke, including launch and asset probes. This is a generated-app release gate, with no Windows archive yet. -8. Tag `v2.0.7`, publish the release, and wait for all archives and checksum - files to upload. +8. Tag `v2.0.7`, publish the release, and wait for the binary and source + archives and checksum files to upload. Compare the published source archive + hash with the reviewed Homebrew formula before pushing the tap. ## 3. Homebrew From 7ff11a07ccb1686c01a82bf11ee76be34ff8d8ad Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 19:30:03 -0400 Subject: [PATCH 09/69] feat(amber-lsp): add --version with the amber_cli release and build commit The fleet gate and setup checks need to prove which amber-lsp build they run; the server previously ignored --version and printed nothing. --- spec/amber_lsp/version_spec.cr | 20 ++++++++++++++++++++ src/amber_lsp.cr | 5 +++++ src/amber_lsp/version.cr | 11 +++++++++++ 3 files changed, 36 insertions(+) create mode 100644 spec/amber_lsp/version_spec.cr diff --git a/spec/amber_lsp/version_spec.cr b/spec/amber_lsp/version_spec.cr new file mode 100644 index 0000000..f22250e --- /dev/null +++ b/spec/amber_lsp/version_spec.cr @@ -0,0 +1,20 @@ +require "./spec_helper" + +describe AmberLSP do + describe ".version_line" do + it "names the server version, the amber_cli release, and the build commit" do + AmberLSP.version_line.should eq( + "amber-lsp #{AmberLSP::VERSION} (amber_cli #{AmberLSP::RELEASE}, commit #{AmberLSP::BUILD_COMMIT})" + ) + end + + it "reads the amber_cli release from shard.yml" do + shard_version = File.read(File.join(__DIR__, "..", "..", "shard.yml"))[/^version:\s*(\S+)/m, 1] + AmberLSP::RELEASE.should eq(shard_version) + end + + it "stamps a git commit when built from a checkout" do + AmberLSP::BUILD_COMMIT.should match(/\A[0-9a-f]{12}\z/) + end + end +end diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 472f055..f1fc322 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -25,4 +25,9 @@ require "./amber_lsp/analyzer" require "./amber_lsp/controller" require "./amber_lsp/server" +if ARGV.includes?("--version") + STDOUT.puts AmberLSP.version_line + exit 0 +end + AmberLSP::Server.new(STDIN, STDOUT).run diff --git a/src/amber_lsp/version.cr b/src/amber_lsp/version.cr index 3767460..2545a00 100644 --- a/src/amber_lsp/version.cr +++ b/src/amber_lsp/version.cr @@ -1,3 +1,14 @@ module AmberLSP VERSION = "1.0.0" + + # The amber_cli release this server ships in, read from shard.yml at compile time. + RELEASE = {{ `shards version "#{__DIR__}/../.."`.chomp.stringify.downcase }} + + # The git commit the binary was built from, or "unknown" outside a git checkout. + BUILD_COMMIT = {{ `git -C "#{__DIR__}" rev-parse --short=12 HEAD 2>/dev/null || echo unknown`.chomp.stringify }} + + # One line that identifies this exact build, printed by `amber-lsp --version`. + def self.version_line : String + "amber-lsp #{VERSION} (amber_cli #{RELEASE}, commit #{BUILD_COMMIT})" + end end From 32acd65c53ed3b3f78a1ab8d1bc478dacafc766e Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 1 Jun 2026 10:33:55 -0400 Subject: [PATCH 10/69] amber-lsp: add FSDD rules (method type signature, doc block, process-manager structure, story grammar) Four diagnostics rules enforcing the FSDD/Amber V2 standards: full method type interfaces, mandatory doc blocks, process-manager structure (typed initialize + perform), and feature-story grammar validity. Build 0/0, 44 specs pass. Co-Authored-By: Claude Opus 4.8 --- .../amber_lsp/fsdd/doc_block_required_spec.cr | 112 +++++++++++++++ .../fsdd/method_type_signature_spec.cr | 103 ++++++++++++++ .../fsdd/process_manager_structure_spec.cr | 130 ++++++++++++++++++ spec/amber_lsp/fsdd/story_grammar_spec.cr | 114 +++++++++++++++ src/amber_lsp.cr | 1 + .../rules/fsdd/doc_block_required_rule.cr | 91 ++++++++++++ .../rules/fsdd/method_type_signature_rule.cr | 92 +++++++++++++ .../fsdd/process_manager_structure_rule.cr | 66 +++++++++ .../rules/fsdd/story_grammar_rule.cr | 101 ++++++++++++++ 9 files changed, 810 insertions(+) create mode 100644 spec/amber_lsp/fsdd/doc_block_required_spec.cr create mode 100644 spec/amber_lsp/fsdd/method_type_signature_spec.cr create mode 100644 spec/amber_lsp/fsdd/process_manager_structure_spec.cr create mode 100644 spec/amber_lsp/fsdd/story_grammar_spec.cr create mode 100644 src/amber_lsp/rules/fsdd/doc_block_required_rule.cr create mode 100644 src/amber_lsp/rules/fsdd/method_type_signature_rule.cr create mode 100644 src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr create mode 100644 src/amber_lsp/rules/fsdd/story_grammar_rule.cr diff --git a/spec/amber_lsp/fsdd/doc_block_required_spec.cr b/spec/amber_lsp/fsdd/doc_block_required_spec.cr new file mode 100644 index 0000000..dd68d95 --- /dev/null +++ b/spec/amber_lsp/fsdd/doc_block_required_spec.cr @@ -0,0 +1,112 @@ +require "../spec_helper" +require "../../../src/amber_lsp/rules/fsdd/doc_block_required_rule" + +describe AmberLSP::Rules::FSDD::DocBlockRequiredRule do + before_each do + AmberLSP::Rules::RuleRegistry.clear + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::DocBlockRequiredRule.new) + end + + describe "#check" do + it "flags a public def without a doc comment" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + diagnostics = rule.check("src/models/user.cr", "def foo : String\nend\n") + diagnostics.size.should eq(1) + diagnostics[0].code.should eq("fsdd/doc-block-required") + diagnostics[0].severity.should eq(AmberLSP::Rules::Severity::Information) + diagnostics[0].message.should contain("foo") + end + + it "flags a public class without a doc comment" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + diagnostics = rule.check("src/models/user.cr", "class User\nend\n") + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("User") + end + + it "stays quiet when a doc comment immediately precedes a public def" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + content = "# Returns a greeting\ndef foo : String\nend\n" + diagnostics = rule.check("src/models/user.cr", content) + diagnostics.should be_empty + end + + it "stays quiet when a doc comment immediately precedes a public class" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + content = "# User model\nclass User\nend\n" + diagnostics = rule.check("src/models/user.cr", content) + diagnostics.should be_empty + end + + it "stays quiet on private def" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + diagnostics = rule.check("src/models/user.cr", "private def foo : String\nend\n") + diagnostics.should be_empty + end + + it "stays quiet on protected def" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + diagnostics = rule.check("src/models/user.cr", "protected def foo : String\nend\n") + diagnostics.should be_empty + end + + it "stays quiet on a def preceded by a standalone private keyword" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + content = "private\n\ndef foo : String\nend\n" + diagnostics = rule.check("src/models/user.cr", content) + diagnostics.should be_empty + end + + it "flags multiple undocumented public methods in one file" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + content = <<-CRYSTAL + def foo : String + "foo" + end + + def bar : Int32 + 42 + end + CRYSTAL + diagnostics = rule.check("src/models/user.cr", content) + diagnostics.size.should eq(2) + end + + it "flags only the undocumented method when another method has a doc comment" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + content = <<-CRYSTAL + # Documented + def good : String + "ok" + end + + def bad : Int32 + 42 + end + CRYSTAL + diagnostics = rule.check("src/models/user.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("bad") + end + + it "stays quiet on an abstract class preceded by a doc comment" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + content = "# Base rule\nabstract class BaseRule\nend\n" + diagnostics = rule.check("src/models/user.cr", content) + diagnostics.should be_empty + end + + it "flags an abstract def without a doc comment" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + diagnostics = rule.check("src/models/user.cr", "abstract def perform : Void\n") + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("perform") + end + + it "produces no diagnostics for an empty file" do + rule = AmberLSP::Rules::FSDD::DocBlockRequiredRule.new + diagnostics = rule.check("src/models/user.cr", "") + diagnostics.should be_empty + end + end +end diff --git a/spec/amber_lsp/fsdd/method_type_signature_spec.cr b/spec/amber_lsp/fsdd/method_type_signature_spec.cr new file mode 100644 index 0000000..408faa7 --- /dev/null +++ b/spec/amber_lsp/fsdd/method_type_signature_spec.cr @@ -0,0 +1,103 @@ +require "../spec_helper" +require "../../../src/amber_lsp/rules/fsdd/method_type_signature_rule" + +describe AmberLSP::Rules::FSDD::MethodTypeSignatureRule do + before_each do + AmberLSP::Rules::RuleRegistry.clear + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new) + end + + describe "#check" do + it "flags a method with an untyped parameter" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + diagnostics = rule.check("src/models/user.cr", "def foo(x)\nend\n") + diagnostics.size.should eq(1) + diagnostics[0].code.should eq("fsdd/method-type-signature") + diagnostics[0].severity.should eq(AmberLSP::Rules::Severity::Warning) + diagnostics[0].message.should contain("foo") + diagnostics[0].message.should contain("untyped parameter") + end + + it "flags a method with typed params but no return type" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + diagnostics = rule.check("src/models/user.cr", "def bar(x : Int32)\nend\n") + diagnostics.size.should eq(1) + diagnostics[0].code.should eq("fsdd/method-type-signature") + diagnostics[0].message.should contain("bar") + diagnostics[0].message.should contain("missing return type") + end + + it "stays quiet on a fully typed method" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + diagnostics = rule.check("src/models/user.cr", "def baz(x : Int32) : String\nend\n") + diagnostics.should be_empty + end + + it "flags a method with no params and no return type" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + diagnostics = rule.check("src/models/user.cr", "def greet\nend\n") + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing return type") + end + + it "stays quiet on a no-param method with return type" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + diagnostics = rule.check("src/models/user.cr", "def count : Int32\nend\n") + diagnostics.should be_empty + end + + it "flags both issues when params are untyped AND return type is missing" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + diagnostics = rule.check("src/services/calculator.cr", "def add(a, b)\nend\n") + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("untyped parameter") + diagnostics[0].message.should contain("missing return type") + end + + it "stays quiet on a method with multiple typed params and return type" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + content = "def add(a : Int32, b : Int32) : Int32\nend\n" + diagnostics = rule.check("src/services/calculator.cr", content) + diagnostics.should be_empty + end + + it "skips initialize methods" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + diagnostics = rule.check("src/models/user.cr", "def initialize(@name : String)\nend\n") + diagnostics.should be_empty + end + + it "flags a method with a mix of typed and untyped params" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + content = "def process(name, count : Int32) : Bool\nend\n" + diagnostics = rule.check("src/models/user.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("untyped parameter") + end + + it "handles multiple methods in one file" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + content = <<-CRYSTAL + def foo(x : Int32) : String + x.to_s + end + + def bar(y) + y + end + + def qux + 42 + end + CRYSTAL + diagnostics = rule.check("src/models/user.cr", content) + diagnostics.size.should eq(2) + end + + it "produces no diagnostics for an empty file" do + rule = AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new + diagnostics = rule.check("src/models/user.cr", "") + diagnostics.should be_empty + end + end +end diff --git a/spec/amber_lsp/fsdd/process_manager_structure_spec.cr b/spec/amber_lsp/fsdd/process_manager_structure_spec.cr new file mode 100644 index 0000000..196c571 --- /dev/null +++ b/spec/amber_lsp/fsdd/process_manager_structure_spec.cr @@ -0,0 +1,130 @@ +require "../spec_helper" +require "../../../src/amber_lsp/rules/fsdd/process_manager_structure_rule" + +describe AmberLSP::Rules::FSDD::ProcessManagerStructureRule do + before_each do + AmberLSP::Rules::RuleRegistry.clear + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new) + end + + describe "#check" do + it "returns empty for files outside process_managers or processes directories" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + diagnostics = rule.check("src/models/user.cr", "class Foo\nend\n") + diagnostics.should be_empty + end + + it "flags a class missing initialize" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + content = <<-CRYSTAL + class Billing::LockAccounts + def perform : Void + end + end + CRYSTAL + diagnostics = rule.check("src/process_managers/billing/lock_accounts.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].code.should eq("fsdd/process-manager-structure") + diagnostics[0].severity.should eq(AmberLSP::Rules::Severity::Warning) + diagnostics[0].message.should contain("LockAccounts") + diagnostics[0].message.should contain("missing initialize with typed parameters") + end + + it "flags a class missing a public perform or call method" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + content = <<-CRYSTAL + class Billing::LockAccounts + def initialize(@account_id : Int32) + end + end + CRYSTAL + diagnostics = rule.check("src/process_managers/billing/lock_accounts.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing public perform or call method") + end + + it "reports both issues when both are missing" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + content = "class Billing::LockAccounts\nend\n" + diagnostics = rule.check("src/process_managers/billing/lock_accounts.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing initialize with typed parameters") + diagnostics[0].message.should contain("missing public perform or call method") + end + + it "stays quiet when class has typed initialize and public perform" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + content = <<-CRYSTAL + class Billing::LockAccounts + def initialize(@account_id : Int32, @reason : String) + end + + def perform : Void + end + end + CRYSTAL + diagnostics = rule.check("src/process_managers/billing/lock_accounts.cr", content) + diagnostics.should be_empty + end + + it "stays quiet when class uses call instead of perform" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + content = <<-CRYSTAL + class Users::CreateUser + def initialize(@email : String) + end + + def call : Void + end + end + CRYSTAL + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.should be_empty + end + + it "flags a class whose initialize has no typed parameters" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + content = <<-CRYSTAL + class Billing::LockAccounts + def initialize(account_id, reason) + end + + def perform : Void + end + end + CRYSTAL + diagnostics = rule.check("src/process_managers/billing/lock_accounts.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing initialize with typed parameters") + end + + it "flags a class whose perform method is private" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + content = <<-CRYSTAL + class Billing::LockAccounts + def initialize(@account_id : Int32) + end + + private def perform : Void + end + end + CRYSTAL + diagnostics = rule.check("src/process_managers/billing/lock_accounts.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing public perform or call method") + end + + it "also applies to files under processes/ directory" do + rule = AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new + content = <<-CRYSTAL + class Billing::RetryPayments + def perform : Void + end + end + CRYSTAL + diagnostics = rule.check("src/processes/billing/retry_payments.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing initialize with typed parameters") + end + end +end diff --git a/spec/amber_lsp/fsdd/story_grammar_spec.cr b/spec/amber_lsp/fsdd/story_grammar_spec.cr new file mode 100644 index 0000000..9157744 --- /dev/null +++ b/spec/amber_lsp/fsdd/story_grammar_spec.cr @@ -0,0 +1,114 @@ +require "../spec_helper" +require "../../../src/amber_lsp/rules/fsdd/story_grammar_rule" + +describe AmberLSP::Rules::FSDD::StoryGrammarRule do + before_each do + AmberLSP::Rules::RuleRegistry.clear + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::StoryGrammarRule.new) + end + + describe "#check" do + it "fires on a story block missing the action verb" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "# As a user\n# views the dashboard\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].code.should eq("fsdd/story-grammar") + diagnostics[0].severity.should eq(AmberLSP::Rules::Severity::Warning) + diagnostics[0].message.should contain("missing action verb") + end + + it "fires on a story block missing the model name" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "# As a user, I want to GET something from the list\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing capitalized Model name") + end + + it "fires on a story block missing both verb and model" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "# As a user\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing action verb") + diagnostics[0].message.should contain("missing capitalized Model name") + end + + it "stays quiet on a complete persona story with GET verb and model" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "# As an Admin, I want to GET Users so I can manage the Account\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.should be_empty + end + + it "stays quiet on a complete persona story with POST verb" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "# As an Admin, I want to POST a new User to the system\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.should be_empty + end + + it "stays quiet on a complete scheduling story using perform verb" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "# At midnight on Monday, perform RetryPayments for all pending Orders\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.should be_empty + end + + it "stays quiet on a recurring story with do verb" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "# Every night, do LockExpiredAccounts for all Users\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.should be_empty + end + + it "stays quiet on a comment block with no story initiator" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "# Private helper that validates inputs\n# Returns nil if invalid\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.should be_empty + end + + it "stays quiet on regular code with no comments" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = "def perform : Void\n validate\nend\n" + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.should be_empty + end + + it "flags only the incomplete story block in a mixed file" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = <<-CRYSTAL + # Regular comment about the class + class Foo + # As a user + def perform : Void + end + end + CRYSTAL + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("missing action verb") + end + + it "stays quiet on a complete multi-line story block" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + content = <<-CRYSTAL + # As an Admin, I want to DELETE a User + # so that I can manage the Account membership + # views the user management page after successful deletion + def destroy : Void + end + CRYSTAL + diagnostics = rule.check("src/process_managers/users/create_user.cr", content) + diagnostics.should be_empty + end + + it "produces no diagnostics for an empty file" do + rule = AmberLSP::Rules::FSDD::StoryGrammarRule.new + diagnostics = rule.check("src/process_managers/users/create_user.cr", "") + diagnostics.should be_empty + end + end +end diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index f1fc322..5bd7408 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -17,6 +17,7 @@ require "./amber_lsp/rules/file_naming/*" require "./amber_lsp/rules/routing/*" require "./amber_lsp/rules/specs/*" require "./amber_lsp/rules/sockets/*" +require "./amber_lsp/rules/fsdd/*" require "./amber_lsp/rules/custom_rule" require "./amber_lsp/document_store" require "./amber_lsp/project_context" diff --git a/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr b/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr new file mode 100644 index 0000000..4754c64 --- /dev/null +++ b/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr @@ -0,0 +1,91 @@ +module AmberLSP::Rules::FSDD + class DocBlockRequiredRule < AmberLSP::Rules::BaseRule + CLASS_RE = /^\s*(abstract\s+)?class\s+/ + DEF_RE = /^\s*(abstract\s+)?def\s+/ + PRIV_RE = /^\s*(private|protected)\s+(abstract\s+)?(def|class)\s+/ + NAME_RE = /\b(?:class|def)\s+(?:self\.)?(\w+[?!]?)/ + + def id : String + "fsdd/doc-block-required" + end + + def description : String + "Every public class and public method must have a preceding doc comment" + end + + def default_severity : AmberLSP::Rules::Severity + Severity::Information + end + + def applies_to : Array(String) + ["src/**"] + end + + def check(file_path : String, content : String) : Array(Diagnostic) + diagnostics = [] of Diagnostic + lines = content.lines + + # prev_nb[i] = index of the previous non-blank line before line i, or -1 + prev_nb = Array(Int32).new(lines.size, -1) + last_non_blank = -1 + lines.each_with_index do |line, i| + prev_nb[i] = last_non_blank + last_non_blank = i unless line.strip.empty? + end + + lines.each_with_index do |line, i| + next if line.strip.empty? + + is_class = CLASS_RE.matches?(line) + is_def = DEF_RE.matches?(line) + next unless is_class || is_def + + # Skip singleton class (class << self) + next if /^\s*class\s+<= 0 + prev_stripped = lines[prev_idx].strip + # Preceded by standalone private / protected + next if prev_stripped == "private" || prev_stripped == "protected" + has_doc = prev_stripped.starts_with?("#") + else + has_doc = false + end + + next if has_doc + + name_match = NAME_RE.match(line) + if nm = name_match + name = nm[1] + start_char = (nm.begin(1) || 0).to_i32 + end_char = (nm.end(1) || line.size).to_i32 + else + name = is_class ? "class" : "method" + start_char = 0 + end_char = 0 + end + + kind = is_class ? "Class" : "Method" + + diagnostics << Diagnostic.new( + range: TextRange.new( + Position.new(i.to_i32, start_char), + Position.new(i.to_i32, end_char) + ), + severity: default_severity, + code: id, + message: "#{kind} '#{name}' is missing a doc comment" + ) + end + + diagnostics + end + end +end + +AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::DocBlockRequiredRule.new) diff --git a/src/amber_lsp/rules/fsdd/method_type_signature_rule.cr b/src/amber_lsp/rules/fsdd/method_type_signature_rule.cr new file mode 100644 index 0000000..6932486 --- /dev/null +++ b/src/amber_lsp/rules/fsdd/method_type_signature_rule.cr @@ -0,0 +1,92 @@ +module AmberLSP::Rules::FSDD + # Crystal lifecycle methods that conventionally omit return type annotations. + SKIP_METHODS = {"initialize", "finalize"} + + class MethodTypeSignatureRule < AmberLSP::Rules::BaseRule + def id : String + "fsdd/method-type-signature" + end + + def description : String + "All method definitions must have fully typed signatures: typed parameters and an explicit return type (FSDD verbose + fully-typed standard)" + end + + def default_severity : AmberLSP::Rules::Severity + Severity::Warning + end + + def applies_to : Array(String) + ["src/**"] + end + + def check(file_path : String, content : String) : Array(Diagnostic) + diagnostics = [] of Diagnostic + + # Group 1: leading whitespace + "def ", Group 2: method name, Group 3: rest of signature + def_pattern = /^(\s*def\s+)(\w+[!?]?)(.*)/ + + content.each_line.with_index do |line, line_number| + match = def_pattern.match(line) + next unless match + + method_name = match[2] + rest = match[3] + + next if SKIP_METHODS.includes?(method_name) + + has_untyped_params = false + has_return_type = false + + if rest.includes?("(") + paren_start = rest.index('(') + paren_end = rest.rindex(')') + + # Multi-line signatures (no closing ) on this line) are skipped — known limitation + next unless paren_start && paren_end && paren_end > paren_start + + params_str = rest[paren_start + 1...paren_end] + after_paren = rest[paren_end + 1..] + + has_return_type = !!(/\s*:\s*\w/.match(after_paren)) + has_untyped_params = untyped_params?(params_str) + else + # No parens: "def foo : ReturnType" or "def foo" (no params) + has_return_type = !!(/\s*:\s*\w/.match(rest)) + end + + issues = [] of String + issues << "untyped parameter(s)" if has_untyped_params + issues << "missing return type" unless has_return_type + next if issues.empty? + + name_start = (match.begin(2) || 0).to_i32 + name_end = (match.end(2) || line.size).to_i32 + + diagnostics << Diagnostic.new( + range: TextRange.new( + Position.new(line_number.to_i32, name_start), + Position.new(line_number.to_i32, name_end) + ), + severity: default_severity, + code: id, + message: "Method '#{method_name}' has incomplete type signature: #{issues.join(", ")}" + ) + end + + diagnostics + end + + private def untyped_params?(params_str : String) : Bool + return false if params_str.strip.empty? + params_str.split(",").any? do |param| + param = param.strip + next false if param.empty? + # Splat/double-splat/block params are exempt (typed differently in Crystal) + next false if param.starts_with?("*") || param.starts_with?("&") + !param.includes?(":") + end + end + end +end + +AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new) diff --git a/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr b/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr new file mode 100644 index 0000000..5c94503 --- /dev/null +++ b/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr @@ -0,0 +1,66 @@ +module AmberLSP::Rules::FSDD + class ProcessManagerStructureRule < AmberLSP::Rules::BaseRule + def id : String + "fsdd/process-manager-structure" + end + + def description : String + "Process manager classes must define an initialize with typed parameters and a public perform or call method" + end + + def default_severity : AmberLSP::Rules::Severity + Severity::Warning + end + + def applies_to : Array(String) + ["src/**/process_managers/**", "src/**/processes/**"] + end + + def check(file_path : String, content : String) : Array(Diagnostic) + return [] of Diagnostic unless file_path.includes?("process_managers/") || file_path.includes?("processes/") + + diagnostics = [] of Diagnostic + lines = content.lines + + # def initialize with at least one typed parameter: matches ( ... : ... ) + init_typed_re = /^\s*def\s+initialize\s*\([^)]*:[^)]*\)/ + # Public def perform or call; private def perform|call is excluded + perform_re = /^\s*def\s+(perform|call)\b/ + private_perform_re = /^\s*private\s+def\s+(perform|call)\b/ + + has_typed_init = lines.any? { |line| init_typed_re.matches?(line) } + has_public_perform = lines.any? do |line| + perform_re.matches?(line) && !private_perform_re.matches?(line) + end + + return [] of Diagnostic if has_typed_init && has_public_perform + + content.each_line.with_index do |line, line_number| + match = /^\s*class\s+(\w[\w:]*)/.match(line) + next unless match + + class_name = match[1] + start_char = (match.begin(1) || 0).to_i32 + end_char = (match.end(1) || line.size).to_i32 + + issues = [] of String + issues << "missing initialize with typed parameters" unless has_typed_init + issues << "missing public perform or call method" unless has_public_perform + + diagnostics << Diagnostic.new( + range: TextRange.new( + Position.new(line_number.to_i32, start_char), + Position.new(line_number.to_i32, end_char) + ), + severity: default_severity, + code: id, + message: "Process manager '#{class_name}': #{issues.join(", ")}" + ) + end + + diagnostics + end + end +end + +AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::ProcessManagerStructureRule.new) diff --git a/src/amber_lsp/rules/fsdd/story_grammar_rule.cr b/src/amber_lsp/rules/fsdd/story_grammar_rule.cr new file mode 100644 index 0000000..67669de --- /dev/null +++ b/src/amber_lsp/rules/fsdd/story_grammar_rule.cr @@ -0,0 +1,101 @@ +module AmberLSP::Rules::FSDD + class StoryGrammarRule < AmberLSP::Rules::BaseRule + STORY_INITIATOR_RE = /\b(As a|At |Every )/ + ACTION_VERB_RE = /\b(GET|POST|PUT|PATCH|DELETE|perform|do)\b/ + STORY_INITIATOR_WORDS = {"As", "At", "Every"} + MODEL_NAME_RE = /\b[A-Z][a-z][a-zA-Z0-9]*\b/ + + def id : String + "fsdd/story-grammar" + end + + def description : String + "Feature story doc comments must include an action verb and a capitalized Model name or process reference" + end + + def default_severity : AmberLSP::Rules::Severity + Severity::Warning + end + + def applies_to : Array(String) + ["src/**"] + end + + def check(file_path : String, content : String) : Array(Diagnostic) + diagnostics = [] of Diagnostic + lines = content.lines + + in_block = false + block_lines = [] of String + block_start = 0 + + lines.each_with_index do |line, i| + is_comment = line.strip.starts_with?("#") + + if is_comment + unless in_block + in_block = true + block_start = i + block_lines = [] of String + end + block_lines << line.strip.lchop("#").strip + elsif in_block + in_block = false + emit_story_diagnostics(block_lines, block_start, diagnostics) + end + end + + emit_story_diagnostics(block_lines, block_start, diagnostics) if in_block + + diagnostics + end + + private def emit_story_diagnostics( + block_lines : Array(String), + block_start : Int32, + diagnostics : Array(Diagnostic) + ) : Nil + block_text = block_lines.join(" ") + return unless STORY_INITIATOR_RE.matches?(block_text) + + issues = [] of String + issues << "missing action verb (GET, POST, PUT, PATCH, DELETE, perform, or do)" unless ACTION_VERB_RE.matches?(block_text) + issues << "missing capitalized Model name or process reference" unless has_model_name?(block_text) + return if issues.empty? + + # Point to the line that contains the story initiator + story_line = block_start + block_lines.each_with_index do |line_content, bi| + if STORY_INITIATOR_RE.matches?(line_content) + story_line = block_start + bi + break + end + end + + diagnostics << Diagnostic.new( + range: TextRange.new( + Position.new(story_line, 0), + Position.new(story_line, 0) + ), + severity: default_severity, + code: id, + message: "Story comment: #{issues.join("; ")}" + ) + end + + # Returns true if text contains a PascalCase word that looks like a model + # or process name (4+ chars, not a story initiator word like "As"/"At"/"Every"). + private def has_model_name?(text : String) : Bool + pos = 0 + while (m = MODEL_NAME_RE.match(text, pos)) + word = m[0] + return true if word.size >= 4 && !STORY_INITIATOR_WORDS.includes?(word) + new_pos = (m.end(0) || pos + 1).to_i32 + pos = new_pos > pos ? new_pos : pos + 1 + end + false + end + end +end + +AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::StoryGrammarRule.new) From a81c40a85381d456762a0c8558eff90edde73a4c Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Sat, 1 Aug 2026 14:36:17 -0400 Subject: [PATCH 11/69] amber-lsp: add scripts/lsp_smoke.rb live-fire stdio proof The spec/amber_lsp/ suite drives AmberLSP::Server in-process over an IO::Memory pair. That proves the code but not the artifact: it cannot catch a stale bin/amber-lsp, a broken build, or a server that goes silent instead of answering. scripts/lsp_smoke.rb spawns the real executable, builds a throwaway Amber-shaped fixture project (shard.yml WITH an `amber` dependency -- ProjectContext.detect keeps the server silent without one), runs a real framed initialize/initialized/didOpen session, and asserts on the textDocument/publishDiagnostics notifications that come back: >= 1 diagnostic for the violating controller, exactly 0 for the clean one. Exit codes are deliberately three-valued: 0 pass, 1 expectations not met, 2 could-not-measure (no binary, handshake failure, timeout). A timeout is never reported as a pass -- a silent server is not a clean server. Ruby 2.6 / stdlib only so it runs under macOS system ruby with no setup. Verified against crystal-alpha 1.21.0 (Crystal 1.21.0 [9c1e8ec64]). --- scripts/lsp_smoke.rb | 290 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 290 insertions(+) create mode 100755 scripts/lsp_smoke.rb diff --git a/scripts/lsp_smoke.rb b/scripts/lsp_smoke.rb new file mode 100755 index 0000000..559fd4e --- /dev/null +++ b/scripts/lsp_smoke.rb @@ -0,0 +1,290 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# lsp_smoke.rb — live-fire proof that the built `amber-lsp` binary really +# speaks LSP over stdio and really publishes diagnostics. +# +# The Crystal specs under spec/amber_lsp/ exercise the server class in-process +# with an IO::Memory pair. That proves the code, not the binary: it cannot +# catch a stale binary on disk, a broken build, or a server that hangs instead +# of answering. This script spawns the ACTUAL executable, drives a real framed +# stdio session against a throwaway Amber-shaped project, and asserts on the +# `textDocument/publishDiagnostics` notifications that come back. +# +# scripts/lsp_smoke.rb # uses ./bin/amber-lsp +# scripts/lsp_smoke.rb --server /path/to/amber-lsp +# scripts/lsp_smoke.rb --timeout 20 --keep # keep the fixture project +# +# Exit 0 = the violating fixture produced >= 1 diagnostic AND the clean fixture +# produced exactly 0. Exit 1 = live-fire expectations not met. Exit 2 = could +# not run at all (no binary, handshake failure, timeout). A timeout is never +# reported as a pass — "I could not measure it" is not "it is clean". +# +# Ruby 2.6 compatible on purpose: it must run under macOS system ruby with no +# gems, so it works anywhere the binary does. + +require "json" +require "fileutils" +require "tmpdir" + +module LSPSmoke + EXIT_OK = 0 + EXIT_FAILED = 1 + EXIT_CANNOT = 2 + + # LSP DiagnosticSeverity + SEVERITY = { 1 => "error", 2 => "warning", 3 => "info", 4 => "hint" }.freeze + + # Reads `Content-Length: N\r\n\r\n` frames off a pipe under a deadline. + # Buffered + non-blocking so a server that goes quiet times out instead of + # wedging the script forever. + class FrameReader + def initialize(io) + @io = io + @buf = String.new.force_encoding(Encoding::BINARY) + end + + def read_frame(deadline) + loop do + frame = take_frame + return frame if frame + return nil unless fill(deadline) + end + end + + private + + def take_frame + idx = @buf.index("\r\n\r\n") + return nil unless idx + + header = @buf[0, idx] + length = header[/Content-Length:\s*(\d+)/i, 1] + raise "malformed LSP header: #{header.inspect}" if length.nil? + + length = length.to_i + total = idx + 4 + length + return nil if @buf.bytesize < total + + body = @buf.byteslice(idx + 4, length) + @buf = @buf.byteslice(total, @buf.bytesize - total) + body.force_encoding(Encoding::UTF_8) + end + + def fill(deadline) + remaining = deadline - Time.now + return false if remaining <= 0 + return false unless IO.select([@io], nil, nil, remaining) + + @buf << @io.read_nonblock(65_536) + true + rescue IO::WaitReadable + true + rescue EOFError, Errno::EIO + false + end + end + + module_function + + def frame(message) + json = JSON.generate(message) + "Content-Length: #{json.bytesize}\r\n\r\n#{json}" + end + + # A minimal project the LSP will actually accept: ProjectContext.detect only + # switches diagnostics on when shard.yml has an `amber` DEPENDENCY. Without + # it the server stays silent and every file looks clean — the exact false + # green this script exists to make impossible. + def build_fixture(dir) + FileUtils.mkdir_p(File.join(dir, "src", "controllers")) + FileUtils.mkdir_p(File.join(dir, "spec", "controllers")) + + File.write(File.join(dir, "shard.yml"), <<~YAML) + name: lsp_smoke_app + version: 0.1.0 + dependencies: + amber: + github: amberframework/amber + YAML + + # Violating: class name does not end in Controller (amber/controller-naming, + # error severity) and the action never renders (amber/action-return-type). + File.write(File.join(dir, "src", "controllers", "users_controller.cr"), <<~CRYSTAL) + class UsersHandler < Amber::Controller::Base + def index + users = ["Alice", "Bob"] + end + end + CRYSTAL + File.write(File.join(dir, "spec", "controllers", "users_controller_spec.cr"), + "# spec placeholder\n") + + # Clean: correct suffix, renders, documented, fully typed. + File.write(File.join(dir, "src", "controllers", "posts_controller.cr"), <<~CRYSTAL) + # Serves the blog post pages. + class PostsController < Amber::Controller::Base + # Renders the list of posts. + def index : String + render("index.ecr") + end + end + CRYSTAL + File.write(File.join(dir, "spec", "controllers", "posts_controller_spec.cr"), + "# spec placeholder\n") + end + + def session(server_bin, dir, timeout) + root_uri = "file://#{dir}" + bad_uri = "file://#{dir}/src/controllers/users_controller.cr" + good_uri = "file://#{dir}/src/controllers/posts_controller.cr" + + messages = [ + frame("jsonrpc" => "2.0", "id" => 1, "method" => "initialize", + "params" => { "rootUri" => root_uri, "capabilities" => {} }), + frame("jsonrpc" => "2.0", "method" => "initialized", "params" => {}), + frame("jsonrpc" => "2.0", "method" => "textDocument/didOpen", + "params" => { "textDocument" => { + "uri" => bad_uri, "languageId" => "crystal", "version" => 1, + "text" => File.read(File.join(dir, "src", "controllers", "users_controller.cr")) + } }), + frame("jsonrpc" => "2.0", "method" => "textDocument/didOpen", + "params" => { "textDocument" => { + "uri" => good_uri, "languageId" => "crystal", "version" => 1, + "text" => File.read(File.join(dir, "src", "controllers", "posts_controller.cr")) + } }), + frame("jsonrpc" => "2.0", "id" => 2, "method" => "shutdown"), + frame("jsonrpc" => "2.0", "method" => "exit") + ] + + published = {} + initialized = false + + io = IO.popen([server_bin], "r+", err: File::NULL) + begin + io.binmode + io.write(messages.join) + io.flush + + reader = FrameReader.new(io) + deadline = Time.now + timeout + + while published.size < 2 || !initialized + raw = reader.read_frame(deadline) + break if raw.nil? + + msg = JSON.parse(raw) + initialized = true if msg["id"] == 1 && msg.key?("result") + next unless msg["method"] == "textDocument/publishDiagnostics" + + published[msg["params"]["uri"]] = msg + end + ensure + begin + io.close + rescue StandardError + nil + end + end + + { initialized: initialized, published: published, bad_uri: bad_uri, good_uri: good_uri } + end + + def describe(diagnostics) + diagnostics.map do |d| + line = d["range"]["start"]["line"].to_i + 1 + sev = SEVERITY.fetch(d["severity"].to_i, d["severity"].to_s) + " #{sev} L#{line} [#{d['code']}] #{d['message']}" + end + end + + def main(argv) + server = File.join(Dir.pwd, "bin", "amber-lsp") + timeout = 15 + keep = false + + until argv.empty? + case (arg = argv.shift) + when "--server" then server = argv.shift.to_s + when "--timeout" then timeout = argv.shift.to_i + when "--keep" then keep = true + when "-h", "--help" + puts "usage: lsp_smoke.rb [--server PATH] [--timeout SECONDS] [--keep]" + return EXIT_OK + else + warn "lsp_smoke: unknown argument #{arg.inspect}" + return EXIT_CANNOT + end + end + + unless File.file?(server) && File.executable?(server) + warn "lsp_smoke: no executable server at #{server}" + warn "lsp_smoke: build it first — CRYSTAL=crystal-alpha shards build amber-lsp" + return EXIT_CANNOT + end + + dir = Dir.mktmpdir("amber_lsp_smoke") + begin + build_fixture(dir) + puts "server: #{server}" + puts "fixture: #{dir}" + puts + + result = session(server, dir, timeout) + + unless result[:initialized] + warn "lsp_smoke: never received a response to `initialize` within #{timeout}s — handshake FAILED." + return EXIT_CANNOT + end + + bad = result[:published][result[:bad_uri]] + good = result[:published][result[:good_uri]] + + if bad.nil? || good.nil? + missing = [] + missing << "violating fixture" if bad.nil? + missing << "clean fixture" if good.nil? + warn "lsp_smoke: no publishDiagnostics for #{missing.join(' and ')} within #{timeout}s." + warn "lsp_smoke: a silent server is NOT a clean server — treating as could-not-measure." + return EXIT_CANNOT + end + + bad_diags = bad["params"]["diagnostics"] + good_diags = good["params"]["diagnostics"] + + puts "--- publishDiagnostics: VIOLATING fixture (src/controllers/users_controller.cr) ---" + puts JSON.pretty_generate(bad) + puts describe(bad_diags) + puts + puts "--- publishDiagnostics: CLEAN fixture (src/controllers/posts_controller.cr) ---" + puts JSON.pretty_generate(good) + puts describe(good_diags) + puts + + ok = true + if bad_diags.empty? + warn "FAIL: violating fixture produced 0 diagnostics (expected >= 1)." + ok = false + end + unless good_diags.empty? + warn "FAIL: clean fixture produced #{good_diags.size} diagnostic(s) (expected 0)." + ok = false + end + + if ok + puts "LIVE-FIRE OK: violating=#{bad_diags.size} diagnostic(s), clean=0." + EXIT_OK + else + EXIT_FAILED + end + ensure + FileUtils.rm_rf(dir) unless keep + puts "fixture kept at #{dir}" if keep + end + rescue StandardError => e + warn "lsp_smoke: could not run (#{e.class}: #{e.message})" + EXIT_CANNOT + end +end + +exit LSPSmoke.main(ARGV) if $PROGRAM_NAME == __FILE__ From c08a8c16c24ba701a85844612150523182adc695 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Sat, 1 Aug 2026 14:57:14 -0400 Subject: [PATCH 12/69] spec: the e2e "fixed" fixture must satisfy OUR OWN FSDD rules (dogfood ruling) agent_e2e_spec asserted that after the agent fixes the file, diagnostics are empty. They were not: the fixture drew 2x fsdd/doc-block-required and 1x fsdd/method-type-signature. This was never Crystal 1.21 drift -- the spec last changed in 06cdee5 and the FSDD rules landed later in 8781d5b, so the new rules correctly fire on a fixture written before they existed. Two ways to make it green: turn the FSDD rules off by default, or fix the fixture. Owner's standing directive decides it -- we need to be eating our own dog food. The rules STAY ON BY DEFAULT; the fixture was wrong, not the rules. So the corrected code now carries a doc comment on the class and on the action and an explicit return type, and "clean" in this spec means clean by our own conventions rather than merely free of the two violations the fixture was originally written to demonstrate. Anything less and this spec would quietly assert that our own conventions are optional. 253 examples, 0 failures, 0 errors, 0 pending --- spec/amber_lsp/integration/agent_e2e_spec.cr | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/spec/amber_lsp/integration/agent_e2e_spec.cr b/spec/amber_lsp/integration/agent_e2e_spec.cr index 912b3a7..213269e 100644 --- a/spec/amber_lsp/integration/agent_e2e_spec.cr +++ b/spec/amber_lsp/integration/agent_e2e_spec.cr @@ -121,9 +121,18 @@ describe "Agent E2E: LSP diagnostic feedback loop" do # The corrected code: # - Renamed "UsersHandler" → "UsersController" (fixes controller-naming) # - Added render call in index (fixes action-return-type) + # - Doc comment on the class and the action (fixes fsdd/doc-block-required) + # - Explicit return type on the action (fixes fsdd/method-type-signature) + # + # The FSDD rules are on by default and they apply to us too. "Clean" in + # this spec means clean by OUR OWN conventions, not merely free of the two + # violations the fixture was originally written to demonstrate — otherwise + # this spec would quietly assert that our conventions are optional. fixed_code = <<-CRYSTAL + # Serves the user-facing account pages. class UsersController < Amber::Controller::Base - def index + # Renders the list of users. + def index : String users = ["Alice", "Bob"] render("index.ecr") end From 66ecb4aebb5228f8b9f03fcd0763fefeca4b9266 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 19:58:49 -0400 Subject: [PATCH 13/69] feat(amber-lsp): parse typed messages and handle full-document didChange --- spec/amber_lsp/controller_spec.cr | 45 +++++ spec/amber_lsp/diagnostic_spec.cr | 8 +- src/amber_lsp/analyzer.cr | 2 +- src/amber_lsp/controller.cr | 290 ++++++++++++++++++++++-------- src/amber_lsp/rules/diagnostic.cr | 61 +++++-- 5 files changed, 307 insertions(+), 99 deletions(-) diff --git a/spec/amber_lsp/controller_spec.cr b/spec/amber_lsp/controller_spec.cr index fb78e2e..c4c0c4f 100644 --- a/spec/amber_lsp/controller_spec.cr +++ b/spec/amber_lsp/controller_spec.cr @@ -1,4 +1,5 @@ require "./spec_helper" +require "../../src/amber_lsp/rules/controllers/naming_rule" describe AmberLSP::Controller do describe "#handle initialize" do @@ -39,6 +40,50 @@ describe AmberLSP::Controller do end end + describe "#handle didChange" do + it "analyzes full-document unsaved content after didOpen" do + with_tempdir do |project| + Dir.mkdir_p(File.join(project, "src/controllers")) + File.write(File.join(project, "shard.yml"), <<-YAML) + name: amber_lsp_controller_spec + version: 0.1.0 + dependencies: + amber: + github: amberframework/amber + YAML + + file_path = File.join(project, "src/controllers/users_controller.cr") + clean_content = <<-CRYSTAL + # Serves the user pages. + class UsersController < Amber::Controller::Base + end + CRYSTAL + File.write(file_path, clean_content) + uri = "file://#{file_path}" + + messages = [ + {"jsonrpc" => "2.0", "id" => 1, "method" => "initialize", "params" => {"rootUri" => "file://#{project}"}}, + {"jsonrpc" => "2.0", "method" => "initialized", "params" => {} of String => String}, + {"jsonrpc" => "2.0", "method" => "textDocument/didOpen", "params" => {"textDocument" => {"uri" => uri, "text" => clean_content}}}, + {"jsonrpc" => "2.0", "method" => "textDocument/didChange", "params" => {"textDocument" => {"uri" => uri, "version" => 2}, "contentChanges" => [{"text" => clean_content.gsub("UsersController", "UsersHandler")}]}}, + {"jsonrpc" => "2.0", "id" => 2, "method" => "shutdown"}, + {"jsonrpc" => "2.0", "method" => "exit"}, + ] + + responses = run_lsp_session(messages) + diagnostics_notifications = responses.select do |response| + response["method"]?.try(&.as_s?) == "textDocument/publishDiagnostics" + end + + diagnostics_notifications.size.should eq(2) + diagnostics_notifications[0]["params"]["diagnostics"].as_a.should be_empty + changed_diagnostics = diagnostics_notifications[1]["params"]["diagnostics"].as_a + changed_diagnostics.map { |diagnostic| diagnostic["code"].as_s }.should contain("amber/controller-naming") + File.read(file_path).should eq(clean_content) + end + end + end + describe "#handle shutdown" do it "returns null result" do input = IO::Memory.new("") diff --git a/spec/amber_lsp/diagnostic_spec.cr b/spec/amber_lsp/diagnostic_spec.cr index d533bf9..cb6889d 100644 --- a/spec/amber_lsp/diagnostic_spec.cr +++ b/spec/amber_lsp/diagnostic_spec.cr @@ -1,8 +1,8 @@ require "./spec_helper" describe AmberLSP::Rules::Diagnostic do - describe "#to_lsp_json" do - it "returns a hash with correct LSP structure" do + describe "#to_lsp_diagnostic" do + it "serializes with the correct LSP structure" do diagnostic = AmberLSP::Rules::Diagnostic.new( range: AmberLSP::Rules::TextRange.new( AmberLSP::Rules::Position.new(5, 10), @@ -13,7 +13,7 @@ describe AmberLSP::Rules::Diagnostic do message: "This is a test diagnostic" ) - json = diagnostic.to_lsp_json + json = JSON.parse(diagnostic.to_lsp_diagnostic.to_json) range = json["range"] range["start"]["line"].as_i.should eq(5) @@ -39,7 +39,7 @@ describe AmberLSP::Rules::Diagnostic do source: "custom-source" ) - json = diagnostic.to_lsp_json + json = JSON.parse(diagnostic.to_lsp_diagnostic.to_json) json["source"].as_s.should eq("custom-source") end end diff --git a/src/amber_lsp/analyzer.cr b/src/amber_lsp/analyzer.cr index 58186a4..50f8238 100644 --- a/src/amber_lsp/analyzer.cr +++ b/src/amber_lsp/analyzer.cr @@ -53,7 +53,7 @@ module AmberLSP severity = @configuration.rule_severity(rule.id, rule.default_severity) rule_diagnostics.each do |diagnostic| - if diagnostic.severity != severity + if diagnostic.severity == rule.default_severity && diagnostic.severity != severity diagnostics << Rules::Diagnostic.new( range: diagnostic.range, severity: severity, diff --git a/src/amber_lsp/controller.cr b/src/amber_lsp/controller.cr index 328ead6..7e2e032 100644 --- a/src/amber_lsp/controller.cr +++ b/src/amber_lsp/controller.cr @@ -2,6 +2,173 @@ require "json" require "uri" module AmberLSP + struct IncomingTextDocument + include JSON::Serializable + + getter uri : String? = nil + getter text : String? = nil + end + + struct IncomingContentChange + include JSON::Serializable + + getter text : String + end + + struct IncomingParams + include JSON::Serializable + + @[JSON::Field(key: "rootUri")] + getter root_uri : String? = nil + @[JSON::Field(key: "rootPath")] + getter root_path : String? = nil + @[JSON::Field(key: "textDocument")] + getter text_document : IncomingTextDocument? = nil + @[JSON::Field(key: "contentChanges")] + getter content_changes : Array(IncomingContentChange)? = nil + getter text : String? = nil + end + + struct IncomingMessage + include JSON::Serializable + + getter method : String? = nil + getter id : Int64 | String | Nil = nil + getter params : IncomingParams? = nil + end + + # :nodoc: + struct SaveOptions + include JSON::Serializable + + @[JSON::Field(key: "includeText")] + getter include_text : Bool = true + + def initialize + end + end + + # :nodoc: + struct TextDocumentSyncOptions + include JSON::Serializable + + @[JSON::Field(key: "openClose")] + getter open_close : Bool = true + getter change : Int32 = 1 + getter save : SaveOptions + + def initialize(@save : SaveOptions = SaveOptions.new) + end + end + + # :nodoc: + struct ServerCapabilities + include JSON::Serializable + + @[JSON::Field(key: "textDocumentSync")] + getter text_document_sync : TextDocumentSyncOptions + + def initialize(@text_document_sync : TextDocumentSyncOptions = TextDocumentSyncOptions.new) + end + end + + # :nodoc: + struct ServerInfo + include JSON::Serializable + + getter name : String = "amber-lsp" + getter version : String = AmberLSP::VERSION + + def initialize + end + end + + # :nodoc: + struct InitializeResult + include JSON::Serializable + + getter capabilities : ServerCapabilities + @[JSON::Field(key: "serverInfo")] + getter server_info : ServerInfo + + def initialize(@capabilities : ServerCapabilities = ServerCapabilities.new, @server_info : ServerInfo = ServerInfo.new) + end + end + + # :nodoc: + struct InitializeResponse + include JSON::Serializable + + getter jsonrpc : String = "2.0" + @[JSON::Field(emit_null: true)] + getter id : Int64 | String | Nil + getter result : InitializeResult + + def initialize(@id : Int64 | String | Nil, @result : InitializeResult = InitializeResult.new) + end + end + + # :nodoc: + struct ShutdownResponse + include JSON::Serializable + + getter jsonrpc : String = "2.0" + @[JSON::Field(emit_null: true)] + getter id : Int64 | String | Nil + @[JSON::Field(emit_null: true)] + getter result : Nil = nil + + def initialize(@id : Int64 | String | Nil) + end + end + + # :nodoc: + struct JsonRpcError + include JSON::Serializable + + getter code : Int32 + getter message : String + + def initialize(@code : Int32, @message : String) + end + end + + # :nodoc: + struct ErrorResponse + include JSON::Serializable + + getter jsonrpc : String = "2.0" + @[JSON::Field(emit_null: true)] + getter id : Int64 | String | Nil + getter error : JsonRpcError + + def initialize(@id : Int64 | String | Nil, @error : JsonRpcError) + end + end + + # :nodoc: + struct PublishDiagnosticsParams + include JSON::Serializable + + getter uri : String + getter diagnostics : Array(Rules::LSPDiagnostic) + + def initialize(@uri : String, @diagnostics : Array(Rules::LSPDiagnostic)) + end + end + + # :nodoc: + struct PublishDiagnosticsNotification + include JSON::Serializable + + getter jsonrpc : String = "2.0" + getter method : String = "textDocument/publishDiagnostics" + getter params : PublishDiagnosticsParams + + def initialize(@params : PublishDiagnosticsParams) + end + end + class Controller @project_context : ProjectContext? = nil @@ -11,23 +178,26 @@ module AmberLSP end def handle(raw_message : String, server : Server) : String? - json = JSON.parse(raw_message) - method = json["method"]?.try(&.as_s) - id = json["id"]? + message = IncomingMessage.from_json(raw_message) + method = message.method + id = message.id case method when "initialize" - handle_initialize(id, json) + handle_initialize(id, message.params) when "initialized" handle_initialized when "textDocument/didOpen" - handle_did_open(json, server) + handle_did_open(message.params, server) + nil + when "textDocument/didChange" + handle_did_change(message.params, server) nil when "textDocument/didSave" - handle_did_save(json, server) + handle_did_save(message.params, server) nil when "textDocument/didClose" - handle_did_close(json, server) + handle_did_close(message.params, server) nil when "shutdown" handle_shutdown(id) @@ -42,18 +212,18 @@ module AmberLSP end end rescue ex : JSON::ParseException - error_response(JSON::Any.new(nil), -32700, "Parse error: #{ex.message}") + error_response(nil, -32700, "Parse error: #{ex.message}") end - private def handle_initialize(id : JSON::Any?, json : JSON::Any) : String - if params = json["params"]? - if root_uri = params["rootUri"]?.try(&.as_s?) + private def handle_initialize(id : Int64 | String | Nil, params : IncomingParams?) : String + if params + if root_uri = params.root_uri root_path = uri_to_path(root_uri) @project_context = ProjectContext.detect(root_path) if ctx = @project_context @analyzer.configure(ctx) end - elsif root_path = params["rootPath"]?.try(&.as_s?) + elsif root_path = params.root_path @project_context = ProjectContext.detect(root_path) if ctx = @project_context @analyzer.configure(ctx) @@ -61,27 +231,7 @@ module AmberLSP end end - result = { - "jsonrpc" => JSON::Any.new("2.0"), - "id" => id || JSON::Any.new(nil), - "result" => JSON::Any.new({ - "capabilities" => JSON::Any.new({ - "textDocumentSync" => JSON::Any.new({ - "openClose" => JSON::Any.new(true), - "change" => JSON::Any.new(1_i64), # Full sync - "save" => JSON::Any.new({ - "includeText" => JSON::Any.new(true), - }), - }), - }), - "serverInfo" => JSON::Any.new({ - "name" => JSON::Any.new("amber-lsp"), - "version" => JSON::Any.new(AmberLSP::VERSION), - }), - }), - } - - result.to_json + InitializeResponse.new(id).to_json end private def handle_initialized : Nil @@ -89,32 +239,40 @@ module AmberLSP nil end - private def handle_did_open(json : JSON::Any, server : Server) : Nil - params = json["params"]? + private def handle_did_open(params : IncomingParams?, server : Server) : Nil return unless params - text_document = params["textDocument"]? + text_document = params.text_document return unless text_document - uri = text_document["uri"]?.try(&.as_s) - text = text_document["text"]?.try(&.as_s) + uri = text_document.uri + text = text_document.text return unless uri && text @document_store.update(uri, text) run_diagnostics(uri, text, server) end - private def handle_did_save(json : JSON::Any, server : Server) : Nil - params = json["params"]? + private def handle_did_change(params : IncomingParams?, server : Server) : Nil return unless params + uri = params.text_document.try(&.uri) + text = params.content_changes.try(&.last?).try(&.text) + return unless uri && text + + @document_store.update(uri, text) + run_diagnostics(uri, text, server) + end - text_document = params["textDocument"]? + private def handle_did_save(params : IncomingParams?, server : Server) : Nil + return unless params + + text_document = params.text_document return unless text_document - uri = text_document["uri"]?.try(&.as_s) + uri = text_document.uri return unless uri - text = params["text"]?.try(&.as_s) + text = params.text if text @document_store.update(uri, text) run_diagnostics(uri, text, server) @@ -123,50 +281,36 @@ module AmberLSP end end - private def handle_did_close(json : JSON::Any, server : Server) : Nil - params = json["params"]? + private def handle_did_close(params : IncomingParams?, server : Server) : Nil return unless params - text_document = params["textDocument"]? + text_document = params.text_document return unless text_document - uri = text_document["uri"]?.try(&.as_s) + uri = text_document.uri return unless uri @document_store.remove(uri) publish_diagnostics(uri, [] of Rules::Diagnostic, server) end - private def handle_shutdown(id : JSON::Any?) : String - result = { - "jsonrpc" => JSON::Any.new("2.0"), - "id" => id || JSON::Any.new(nil), - "result" => JSON::Any.new(nil), - } - result.to_json + private def handle_shutdown(id : Int64 | String | Nil) : String + ShutdownResponse.new(id).to_json end private def handle_exit(server : Server) : Nil server.stop end - private def error_response(id : JSON::Any?, code : Int32, message : String) : String - result = { - "jsonrpc" => JSON::Any.new("2.0"), - "id" => id || JSON::Any.new(nil), - "error" => JSON::Any.new({ - "code" => JSON::Any.new(code.to_i64), - "message" => JSON::Any.new(message), - }), - } - result.to_json + private def error_response(id : Int64 | String | Nil, code : Int32, message : String) : String + ErrorResponse.new(id, JsonRpcError.new(code, message)).to_json end private def run_diagnostics(uri : String, content : String, server : Server) : Nil file_path = uri_to_path(uri) - # Only analyze Crystal files - return unless file_path.ends_with?(".cr") + # Analyze Crystal source and the app-owned performance convention files. + return unless file_path.ends_with?(".cr") || file_path.ends_with?(".ecr") || file_path.ends_with?(".slang") || file_path.ends_with?(File.join("performance", "budget.json")) || file_path.ends_with?(File.join("performance", "opt_out.json")) # Only run if we detected an Amber project ctx = @project_context @@ -177,17 +321,9 @@ module AmberLSP end private def publish_diagnostics(uri : String, diagnostics : Array(Rules::Diagnostic), server : Server) : Nil - lsp_diagnostics = diagnostics.map(&.to_lsp_json) - - notification = { - "jsonrpc" => JSON::Any.new("2.0"), - "method" => JSON::Any.new("textDocument/publishDiagnostics"), - "params" => JSON::Any.new({ - "uri" => JSON::Any.new(uri), - "diagnostics" => JSON::Any.new(lsp_diagnostics.map { |d| JSON::Any.new(d) }), - }), - } - + list_of_lsp_diagnostics = diagnostics.map(&.to_lsp_diagnostic) + params = PublishDiagnosticsParams.new(uri, list_of_lsp_diagnostics) + notification = PublishDiagnosticsNotification.new(params) server.write_notification(notification.to_json) end diff --git a/src/amber_lsp/rules/diagnostic.cr b/src/amber_lsp/rules/diagnostic.cr index d7b4f4a..b4d1a98 100644 --- a/src/amber_lsp/rules/diagnostic.cr +++ b/src/amber_lsp/rules/diagnostic.cr @@ -1,6 +1,43 @@ require "json" module AmberLSP::Rules + # :nodoc: + struct LSPPosition + include JSON::Serializable + + getter line : Int32 + getter character : Int32 + + def initialize(@line : Int32, @character : Int32) + end + end + + # :nodoc: + struct LSPRange + include JSON::Serializable + + getter start : LSPPosition + @[JSON::Field(key: "end")] + getter end_position : LSPPosition + + def initialize(@start : LSPPosition, @end_position : LSPPosition) + end + end + + # :nodoc: + struct LSPDiagnostic + include JSON::Serializable + + getter range : LSPRange + getter severity : Int32 + getter code : String + getter source : String + getter message : String + + def initialize(@range : LSPRange, @severity : Int32, @code : String, @source : String, @message : String) + end + end + struct Position getter line : Int32 getter character : Int32 @@ -33,23 +70,13 @@ module AmberLSP::Rules ) end - def to_lsp_json : Hash(String, JSON::Any) - { - "range" => JSON::Any.new({ - "start" => JSON::Any.new({ - "line" => JSON::Any.new(@range.start.line.to_i64), - "character" => JSON::Any.new(@range.start.character.to_i64), - }), - "end" => JSON::Any.new({ - "line" => JSON::Any.new(@range.end.line.to_i64), - "character" => JSON::Any.new(@range.end.character.to_i64), - }), - }), - "severity" => JSON::Any.new(@severity.value.to_i64), - "code" => JSON::Any.new(@code), - "source" => JSON::Any.new(@source), - "message" => JSON::Any.new(@message), - } + def to_lsp_diagnostic : LSPDiagnostic + lsp_range = LSPRange.new( + LSPPosition.new(@range.start.line, @range.start.character), + LSPPosition.new(@range.end.line, @range.end.character), + ) + + LSPDiagnostic.new(lsp_range, @severity.value, @code, @source, @message) end end end From 8e2ed64f2dd4c24bdf97d142ed9440660d70fd23 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 20:08:36 -0400 Subject: [PATCH 14/69] feat(amber-lsp): report explicit coverage states --- .../analyze_file_with_coverage_spec.cr | 65 +++++++++++++ .../check_file_for_diagnostics_spec.cr | 75 ++++++++++++++- spec/amber_lsp/configuration_spec.cr | 7 +- spec/amber_lsp/controller_spec.cr | 31 ++++++ spec/amber_lsp/spec_helper.cr | 1 + src/amber_lsp.cr | 1 + src/amber_lsp/analyze_file_with_coverage.cr | 62 ++++++++++++ src/amber_lsp/analyzer.cr | 2 - src/amber_lsp/check_file_for_diagnostics.cr | 46 ++++++--- src/amber_lsp/configuration.cr | 2 - src/amber_lsp/controller.cr | 94 +++++++++++++++---- src/amber_lsp/coverage.cr | 7 ++ src/amber_lsp/coverage/covered.cr | 9 ++ src/amber_lsp/coverage/declined.cr | 9 ++ src/amber_lsp/coverage/failed.cr | 9 ++ src/amber_lsp/project_context.cr | 25 ++++- .../rules/fsdd/doc_block_required_rule.cr | 10 +- .../fsdd/process_manager_structure_rule.cr | 4 +- .../rules/fsdd/story_grammar_rule.cr | 10 +- 19 files changed, 414 insertions(+), 55 deletions(-) create mode 100644 spec/amber_lsp/analyze_file_with_coverage_spec.cr create mode 100644 src/amber_lsp/analyze_file_with_coverage.cr create mode 100644 src/amber_lsp/coverage.cr create mode 100644 src/amber_lsp/coverage/covered.cr create mode 100644 src/amber_lsp/coverage/declined.cr create mode 100644 src/amber_lsp/coverage/failed.cr diff --git a/spec/amber_lsp/analyze_file_with_coverage_spec.cr b/spec/amber_lsp/analyze_file_with_coverage_spec.cr new file mode 100644 index 0000000..7282dfd --- /dev/null +++ b/spec/amber_lsp/analyze_file_with_coverage_spec.cr @@ -0,0 +1,65 @@ +require "./spec_helper" +require "../../src/amber_lsp/rules/controllers/naming_rule" +require "../../src/amber_lsp/analyze_file_with_coverage" + +describe AmberLSP::AnalyzeFileWithCoverage do + before_each do + AmberLSP::Rules::RuleRegistry.clear + end + + describe "#perform" do + it "returns Covered with diagnostics for an Amber project" do + with_tempdir do |project| + Dir.mkdir_p(File.join(project, "src/controllers")) + File.write(File.join(project, "shard.yml"), <<-YAML) + name: amber_lsp_coverage_spec + version: 0.1.0 + dependencies: + amber: + github: amberframework/amber + YAML + + file_path = File.join(project, "src/controllers/users_controller.cr") + File.write(file_path, "class UsersHandler < Amber::Controller::Base\nend\n") + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::Controllers::NamingRule.new) + + coverage = AmberLSP::AnalyzeFileWithCoverage.new(file_path).perform + + coverage.should be_a(AmberLSP::Coverage::Covered) + if covered = coverage.as?(AmberLSP::Coverage::Covered) + covered.list_of_diagnostics.map(&.code).should contain("amber/controller-naming") + end + end + end + + it "returns Declined with a reason for a project outside the stack" do + with_tempdir do |project| + File.write(File.join(project, "shard.yml"), "name: plain_crystal_app\nversion: 0.1.0\n") + file_path = File.join(project, "main.cr") + File.write(file_path, "puts \"hello\"\n") + + coverage = AmberLSP::AnalyzeFileWithCoverage.new(file_path).perform + + coverage.should be_a(AmberLSP::Coverage::Declined) + if declined = coverage.as?(AmberLSP::Coverage::Declined) + declined.reason.should contain("stack") + end + end + end + + it "returns Failed with the parse error for an invalid shard manifest" do + with_tempdir do |project| + File.write(File.join(project, "shard.yml"), "{{invalid yaml") + file_path = File.join(project, "main.cr") + File.write(file_path, "puts \"hello\"\n") + + coverage = AmberLSP::AnalyzeFileWithCoverage.new(file_path).perform + + coverage.should be_a(AmberLSP::Coverage::Failed) + if failed = coverage.as?(AmberLSP::Coverage::Failed) + failed.error.should contain("shard.yml") + end + end + end + end +end diff --git a/spec/amber_lsp/check_file_for_diagnostics_spec.cr b/spec/amber_lsp/check_file_for_diagnostics_spec.cr index 220cdc7..fc6f5ed 100644 --- a/spec/amber_lsp/check_file_for_diagnostics_spec.cr +++ b/spec/amber_lsp/check_file_for_diagnostics_spec.cr @@ -3,6 +3,10 @@ require "../../src/amber_lsp/rules/controllers/naming_rule" require "../../src/amber_lsp/check_file_for_diagnostics" describe AmberLSP::CheckFileForDiagnostics do + before_each do + AmberLSP::Rules::RuleRegistry.clear + end + it "prints a file and line for a real rule violation" do with_tempdir do |project| Dir.mkdir_p(File.join(project, "src/controllers")) @@ -10,11 +14,20 @@ describe AmberLSP::CheckFileForDiagnostics do File.write(file_path, "class UsersHandler < Amber::Controller::Base\nend\n") output = IO::Memory.new errors = IO::Memory.new + File.write(File.join(project, "shard.yml"), <<-YAML) + name: amber_lsp_check_spec + version: 0.1.0 + dependencies: + amber: + github: amberframework/amber + YAML previous_directory = Dir.current begin Dir.cd(project) + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::Controllers::NamingRule.new) status = AmberLSP::CheckFileForDiagnostics.new(file_path, output, errors).perform status.should eq(1) + output.to_s.lines[0].should eq("amber-lsp: covered 1 diagnostic, 1 error") output.to_s.should contain("#{file_path}:1: error: amber/controller-naming") errors.to_s.should be_empty ensure @@ -25,10 +38,68 @@ describe AmberLSP::CheckFileForDiagnostics do it "reports a missing file as a check error" do with_tempdir do |project| + output = IO::Memory.new + errors = IO::Memory.new + status = AmberLSP::CheckFileForDiagnostics.new(File.join(project, "missing.cr"), output, errors).perform + status.should eq(2) + output.to_s.should start_with("amber-lsp: failed ") + errors.to_s.should be_empty + end + end + + it "returns zero and covered status for a clean covered file" do + with_tempdir do |project| + Dir.mkdir_p(File.join(project, "src/controllers")) + File.write(File.join(project, "shard.yml"), <<-YAML) + name: amber_lsp_check_spec + version: 0.1.0 + dependencies: + amber: + github: amberframework/amber + YAML + file_path = File.join(project, "src/controllers/posts_controller.cr") + File.write(file_path, "class PostsController < Amber::Controller::Base\nend\n") + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::Controllers::NamingRule.new) + output = IO::Memory.new + errors = IO::Memory.new + + status = AmberLSP::CheckFileForDiagnostics.new(file_path, output, errors).perform + + status.should eq(0) + output.to_s.should eq("amber-lsp: covered no diagnostics\n") + errors.to_s.should be_empty + end + end + + it "returns two and declined status for a project outside the stack" do + with_tempdir do |project| + File.write(File.join(project, "shard.yml"), "name: plain_crystal_app\nversion: 0.1.0\n") + file_path = File.join(project, "main.cr") + File.write(file_path, "puts \"hello\"\n") + output = IO::Memory.new errors = IO::Memory.new - status = AmberLSP::CheckFileForDiagnostics.new(File.join(project, "missing.cr"), IO::Memory.new, errors).perform + + status = AmberLSP::CheckFileForDiagnostics.new(file_path, output, errors).perform + + status.should eq(2) + output.to_s.should eq("amber-lsp: declined project is not an Amber V2 stack project\n") + errors.to_s.should be_empty + end + end + + it "returns two and failed status for an invalid shard manifest" do + with_tempdir do |project| + File.write(File.join(project, "shard.yml"), "{{invalid yaml") + file_path = File.join(project, "main.cr") + File.write(file_path, "puts \"hello\"\n") + output = IO::Memory.new + errors = IO::Memory.new + + status = AmberLSP::CheckFileForDiagnostics.new(file_path, output, errors).perform + status.should eq(2) - errors.to_s.should contain("file not found") + output.to_s.should start_with("amber-lsp: failed shard.yml") + errors.to_s.should be_empty end end end diff --git a/spec/amber_lsp/configuration_spec.cr b/spec/amber_lsp/configuration_spec.cr index 30ab946..def5eef 100644 --- a/spec/amber_lsp/configuration_spec.cr +++ b/spec/amber_lsp/configuration_spec.cr @@ -61,9 +61,10 @@ describe AmberLSP::Configuration do config.exclude_patterns.should eq(["lib/", "tmp/", "db/migrations/"]) end - it "handles invalid YAML gracefully" do - config = AmberLSP::Configuration.parse("{{invalid") - config.rule_enabled?("any-rule").should be_true + it "raises for invalid YAML so coverage can report the failure" do + expect_raises(YAML::ParseException) do + AmberLSP::Configuration.parse("{{invalid") + end end end diff --git a/spec/amber_lsp/controller_spec.cr b/spec/amber_lsp/controller_spec.cr index af56a65..41c497d 100644 --- a/spec/amber_lsp/controller_spec.cr +++ b/spec/amber_lsp/controller_spec.cr @@ -53,6 +53,8 @@ describe AmberLSP::Controller do YAML file_path = File.join(project, "src/controllers/users_controller.cr") + Dir.mkdir_p(File.join(project, "spec/controllers")) + File.write(File.join(project, "spec/controllers/users_controller_spec.cr"), "# Controller specs.\n") clean_content = <<-CRYSTAL # Serves the user pages. class UsersController < Amber::Controller::Base @@ -85,6 +87,35 @@ describe AmberLSP::Controller do end end + describe "#handle coverage status" do + it "reports declined coverage to the LSP client" do + with_tempdir do |project| + File.write(File.join(project, "shard.yml"), "name: plain_crystal_app\nversion: 0.1.0\n") + file_path = File.join(project, "main.cr") + File.write(file_path, "puts \"hello\"\n") + content = File.read(file_path) + + messages = [ + {"jsonrpc" => "2.0", "id" => 1, "method" => "initialize", "params" => {"rootUri" => "file://#{project}"}}, + {"jsonrpc" => "2.0", "method" => "initialized", "params" => {} of String => String}, + {"jsonrpc" => "2.0", "method" => "textDocument/didOpen", "params" => {"textDocument" => {"uri" => "file://#{file_path}", "text" => content}}}, + {"jsonrpc" => "2.0", "id" => 2, "method" => "shutdown"}, + {"jsonrpc" => "2.0", "method" => "exit"}, + ] + + responses = run_lsp_session(messages) + coverage_messages = responses.select do |response| + response["method"]?.try(&.as_s?) == "window/logMessage" + end + + coverage_messages.size.should eq(1) + coverage_messages[0]["params"]["message"].as_s.should eq( + "amber-lsp: declined project is not an Amber V2 stack project" + ) + end + end + end + describe "#handle shutdown" do it "returns null result" do input = IO::Memory.new("") diff --git a/spec/amber_lsp/spec_helper.cr b/spec/amber_lsp/spec_helper.cr index 6a2a665..b7d488a 100644 --- a/spec/amber_lsp/spec_helper.cr +++ b/spec/amber_lsp/spec_helper.cr @@ -24,6 +24,7 @@ require "../../src/amber_lsp/library_rule_packs/grant_tenancy/visit_grant_schema require "../../src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs" require "../../src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts" require "../../src/amber_lsp/analyzer" +require "../../src/amber_lsp/analyze_file_with_coverage" require "../../src/amber_lsp/controller" require "../../src/amber_lsp/server" diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 2349148..5631d47 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -37,6 +37,7 @@ require "./amber_lsp/library_rule_packs/grant_tenancy/visit_grant_schema_queries require "./amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs" require "./amber_lsp/library_rule_packs/print_detected_rule_pack_contexts" require "./amber_lsp/analyzer" +require "./amber_lsp/analyze_file_with_coverage" require "./amber_lsp/controller" require "./amber_lsp/server" require "./amber_lsp/check_file_for_diagnostics" diff --git a/src/amber_lsp/analyze_file_with_coverage.cr b/src/amber_lsp/analyze_file_with_coverage.cr new file mode 100644 index 0000000..fdc68e4 --- /dev/null +++ b/src/amber_lsp/analyze_file_with_coverage.cr @@ -0,0 +1,62 @@ +require "./coverage" + +module AmberLSP + # When a Crystal file, its project root, and optional current analysis state + # are provided, then return diagnostics with an explicit coverage outcome. + class AnalyzeFileWithCoverage + getter analysis_analyzer : Analyzer? + + def initialize( + @file_path : String, + @content : String? = nil, + @project_root_path : String? = nil, + @configured_analyzer : Analyzer? = nil, + ) + @analysis_analyzer = @configured_analyzer + end + + def perform : Coverage::Result + current_content = document_content + current_project_context = detect_project_context + if error = current_project_context.failure_reason + return Coverage::Failed.new(error) + end + + unless current_project_context.amber_project? + return Coverage::Declined.new("project is not an Amber V2 stack project") + end + + analyzer = analyzer_for(current_project_context) + @analysis_analyzer = analyzer + Coverage::Covered.new(analyzer.analyze(@file_path, current_content)) + rescue ex : Exception + Coverage::Failed.new("#{ex.class}: #{ex.message}") + end + + private def detect_project_context : ProjectContext + if root_path = @project_root_path + ProjectContext.detect(root_path) + else + ProjectContext.detect_for_file(@file_path) + end + end + + private def analyzer_for(project_context : ProjectContext) : Analyzer + if analyzer = @configured_analyzer + return analyzer + end + + analyzer = Analyzer.new + analyzer.configure(project_context) + analyzer + end + + private def document_content : String + if content = @content + return content + end + + File.read(@file_path) + end + end +end diff --git a/src/amber_lsp/analyzer.cr b/src/amber_lsp/analyzer.cr index 88c7b09..14f13ce 100644 --- a/src/amber_lsp/analyzer.cr +++ b/src/amber_lsp/analyzer.cr @@ -53,8 +53,6 @@ module AmberLSP ) Rules::RuleRegistry.register(rule) end - rescue ex - STDERR.puts "WARNING: Failed to load custom rules: #{ex.message}" end def analyze(file_path : String, content : String) : Array(Rules::Diagnostic) diff --git a/src/amber_lsp/check_file_for_diagnostics.cr b/src/amber_lsp/check_file_for_diagnostics.cr index c962fd0..3b04c26 100644 --- a/src/amber_lsp/check_file_for_diagnostics.cr +++ b/src/amber_lsp/check_file_for_diagnostics.cr @@ -1,26 +1,44 @@ module AmberLSP class CheckFileForDiagnostics - def initialize(@file_path : String, @output : IO = STDOUT, @error : IO = STDERR) + def initialize(@file_path : String, @output : IO = STDOUT, _error : IO = STDERR) end def perform : Int32 - path = File.expand_path(@file_path) - unless File.file?(path) - @error.puts "amber-lsp: file not found: #{path}" - return 2 + file_path = File.expand_path(@file_path) + analysis = AnalyzeFileWithCoverage.new(file_path).perform + + if analysis.is_a?(Coverage::Covered) + return print_covered_result(analysis, file_path) end - analyzer = Analyzer.new - analyzer.configure(ProjectContext.detect(Dir.current)) - diagnostics = analyzer.analyze(path, File.read(path)) - diagnostics.each do |diagnostic| - line = diagnostic.range.start.line + 1 - @output.puts "#{path}:#{line}: #{diagnostic.severity.to_s.downcase}: #{diagnostic.code}: #{diagnostic.message}" + if analysis.is_a?(Coverage::Declined) + @output.puts "amber-lsp: declined #{analysis.reason}" + elsif analysis.is_a?(Coverage::Failed) + @output.puts "amber-lsp: failed #{analysis.error}" end - diagnostics.any? { |diagnostic| diagnostic.severity == Rules::Severity::Error } ? 1 : 0 - rescue ex : File::Error | YAML::ParseException - @error.puts "amber-lsp: #{ex.message}" + 2 end + + private def print_covered_result(covered : Coverage::Covered, file_path : String) : Int32 + list_of_diagnostics = covered.list_of_diagnostics + error_count = list_of_diagnostics.count { |diagnostic| diagnostic.severity == Rules::Severity::Error } + @output.puts covered_status_line(list_of_diagnostics.size, error_count) + + list_of_diagnostics.each do |diagnostic| + line = diagnostic.range.start.line + 1 + @output.puts "#{file_path}:#{line}: #{diagnostic.severity.to_s.downcase}: #{diagnostic.code}: #{diagnostic.message}" + end + + error_count.zero? ? 0 : 1 + end + + private def covered_status_line(diagnostic_count : Int32, error_count : Int32) : String + return "amber-lsp: covered no diagnostics" if diagnostic_count.zero? + + diagnostic_word = diagnostic_count == 1 ? "diagnostic" : "diagnostics" + error_word = error_count == 1 ? "error" : "errors" + "amber-lsp: covered #{diagnostic_count} #{diagnostic_word}, #{error_count} #{error_word}" + end end end diff --git a/src/amber_lsp/configuration.cr b/src/amber_lsp/configuration.cr index 2b95cb9..b1382f5 100644 --- a/src/amber_lsp/configuration.cr +++ b/src/amber_lsp/configuration.cr @@ -116,8 +116,6 @@ module AmberLSP exclude_patterns: exclude_patterns, custom_rules: custom_rules, ) - rescue YAML::ParseException - Configuration.new end def rule_enabled?(id : String) : Bool diff --git a/src/amber_lsp/controller.cr b/src/amber_lsp/controller.cr index 14d7cc8..e2a7062 100644 --- a/src/amber_lsp/controller.cr +++ b/src/amber_lsp/controller.cr @@ -1,4 +1,5 @@ require "json" +require "set" require "uri" module AmberLSP @@ -169,12 +170,37 @@ module AmberLSP end end + # :nodoc: + struct LogMessageParams + include JSON::Serializable + + getter type : Int32 + getter message : String + + def initialize(@type : Int32, @message : String) + end + end + + # :nodoc: + struct LogMessageNotification + include JSON::Serializable + + getter jsonrpc : String = "2.0" + getter method : String = "window/logMessage" + getter params : LogMessageParams + + def initialize(@params : LogMessageParams) + end + end + class Controller - @project_context : ProjectContext? = nil + @project_root_path : String? = nil + @analyzer : Analyzer? = nil + @last_coverage_status : String? = nil + @uris_with_current_diagnostics = Set(String).new def initialize @document_store = DocumentStore.new - @analyzer = Analyzer.new end def handle(raw_message : String, server : Server) : String? @@ -218,16 +244,9 @@ module AmberLSP private def handle_initialize(id : Int64 | String | Nil, params : IncomingParams?) : String if params if root_uri = params.root_uri - root_path = uri_to_path(root_uri) - @project_context = ProjectContext.detect(root_path) - if ctx = @project_context - @analyzer.configure(ctx) - end + set_project_root_path(uri_to_path(root_uri)) elsif root_path = params.root_path - @project_context = ProjectContext.detect(root_path) - if ctx = @project_context - @analyzer.configure(ctx) - end + set_project_root_path(root_path) end end @@ -239,6 +258,10 @@ module AmberLSP nil end + private def set_project_root_path(project_root_path : String) : Nil + @project_root_path = project_root_path + end + private def handle_did_open(params : IncomingParams?, server : Server) : Nil return unless params @@ -291,7 +314,10 @@ module AmberLSP return unless uri @document_store.remove(uri) - publish_diagnostics(uri, [] of Rules::Diagnostic, server) + if @uris_with_current_diagnostics.includes?(uri) + publish_diagnostics(uri, [] of Rules::Diagnostic, server) + @uris_with_current_diagnostics.delete(uri) + end end private def handle_shutdown(id : Int64 | String | Nil) : String @@ -310,14 +336,46 @@ module AmberLSP file_path = uri_to_path(uri) # Analyze Crystal source and the app-owned performance convention files. - return unless file_path.ends_with?(".cr") || file_path.ends_with?(".ecr") || file_path.ends_with?(".slang") || file_path.ends_with?(File.join("performance", "budget.json")) || file_path.ends_with?(File.join("performance", "opt_out.json")) + is_supported_analysis_file = file_path.ends_with?(".cr") || + file_path.ends_with?(".ecr") || + file_path.ends_with?(".slang") || + file_path.ends_with?(File.join("performance", "budget.json")) || + file_path.ends_with?(File.join("performance", "opt_out.json")) + return unless is_supported_analysis_file + + analysis = AnalyzeFileWithCoverage.new( + file_path, + content, + @project_root_path, + @analyzer, + ) + coverage = analysis.perform + @analyzer = analysis.analysis_analyzer + + if coverage.is_a?(Coverage::Covered) + publish_diagnostics(uri, coverage.list_of_diagnostics, server) + @uris_with_current_diagnostics.add(uri) + @last_coverage_status = nil + elsif coverage.is_a?(Coverage::Declined) + status_line = "amber-lsp: declined #{coverage.reason}" + publish_coverage_status(uri, status_line, 2, server) + elsif coverage.is_a?(Coverage::Failed) + status_line = "amber-lsp: failed #{coverage.error}" + publish_coverage_status(uri, status_line, 1, server) + end + end - ctx = @project_context - return unless ctx - return unless ctx.amber_project? || @analyzer.has_applicable_library_rule_pack?(file_path, content) + private def publish_coverage_status(uri : String, status_line : String, message_type : Int32, server : Server) : Nil + return if @last_coverage_status == status_line - diagnostics = @analyzer.analyze(file_path, content) - publish_diagnostics(uri, diagnostics, server) + if @uris_with_current_diagnostics.includes?(uri) + publish_diagnostics(uri, [] of Rules::Diagnostic, server) + @uris_with_current_diagnostics.delete(uri) + end + params = LogMessageParams.new(message_type, status_line) + notification = LogMessageNotification.new(params) + server.write_notification(notification.to_json) + @last_coverage_status = status_line end private def publish_diagnostics(uri : String, diagnostics : Array(Rules::Diagnostic), server : Server) : Nil diff --git a/src/amber_lsp/coverage.cr b/src/amber_lsp/coverage.cr new file mode 100644 index 0000000..c19b0f0 --- /dev/null +++ b/src/amber_lsp/coverage.cr @@ -0,0 +1,7 @@ +require "./coverage/covered" +require "./coverage/declined" +require "./coverage/failed" + +module AmberLSP::Coverage + alias Result = Covered | Declined | Failed +end diff --git a/src/amber_lsp/coverage/covered.cr b/src/amber_lsp/coverage/covered.cr new file mode 100644 index 0000000..ff5e5cc --- /dev/null +++ b/src/amber_lsp/coverage/covered.cr @@ -0,0 +1,9 @@ +module AmberLSP::Coverage + # :nodoc: + struct Covered + getter list_of_diagnostics : Array(AmberLSP::Rules::Diagnostic) + + def initialize(@list_of_diagnostics : Array(AmberLSP::Rules::Diagnostic)) + end + end +end diff --git a/src/amber_lsp/coverage/declined.cr b/src/amber_lsp/coverage/declined.cr new file mode 100644 index 0000000..438f7b0 --- /dev/null +++ b/src/amber_lsp/coverage/declined.cr @@ -0,0 +1,9 @@ +module AmberLSP::Coverage + # :nodoc: + struct Declined + getter reason : String + + def initialize(@reason : String) + end + end +end diff --git a/src/amber_lsp/coverage/failed.cr b/src/amber_lsp/coverage/failed.cr new file mode 100644 index 0000000..c9b44cc --- /dev/null +++ b/src/amber_lsp/coverage/failed.cr @@ -0,0 +1,9 @@ +module AmberLSP::Coverage + # :nodoc: + struct Failed + getter error : String + + def initialize(@error : String) + end + end +end diff --git a/src/amber_lsp/project_context.cr b/src/amber_lsp/project_context.cr index 4b550ee..f7eebc7 100644 --- a/src/amber_lsp/project_context.cr +++ b/src/amber_lsp/project_context.cr @@ -5,11 +5,13 @@ module AmberLSP getter root_path : String getter? amber_project : Bool getter shard_name : String? + getter failure_reason : String? def initialize( @root_path : String, @amber_project : Bool = false, @shard_name : String? = nil, + @failure_reason : String? = nil, ) end @@ -29,8 +31,27 @@ module AmberLSP amber_project: is_amber, shard_name: shard_name, ) - rescue YAML::ParseException - ProjectContext.new(root_path, amber_project: false) + rescue ex : YAML::ParseException + ProjectContext.new(root_path, failure_reason: "shard.yml is invalid: #{ex.message}") + rescue ex : File::Error + ProjectContext.new(root_path, failure_reason: "shard.yml could not be read: #{ex.message}") + end + + def self.detect_for_file(file_path : String) : ProjectContext + detect(project_root_for_file(file_path)) + end + + def self.project_root_for_file(file_path : String) : String + current_path = File.expand_path(file_path) + current_path = File.dirname(current_path) unless File.directory?(current_path) + + loop do + return current_path if File.file?(File.join(current_path, "shard.yml")) + + parent_path = File.dirname(current_path) + return current_path if parent_path == current_path + current_path = parent_path + end end private def self.has_amber_dependency?(shard_configuration : YAML::Any) : Bool diff --git a/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr b/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr index 4754c64..72bb190 100644 --- a/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr +++ b/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr @@ -37,7 +37,7 @@ module AmberLSP::Rules::FSDD next if line.strip.empty? is_class = CLASS_RE.matches?(line) - is_def = DEF_RE.matches?(line) + is_def = DEF_RE.matches?(line) next unless is_class || is_def # Skip singleton class (class << self) @@ -61,13 +61,13 @@ module AmberLSP::Rules::FSDD name_match = NAME_RE.match(line) if nm = name_match - name = nm[1] + name = nm[1] start_char = (nm.begin(1) || 0).to_i32 - end_char = (nm.end(1) || line.size).to_i32 + end_char = (nm.end(1) || line.size).to_i32 else - name = is_class ? "class" : "method" + name = is_class ? "class" : "method" start_char = 0 - end_char = 0 + end_char = 0 end kind = is_class ? "Class" : "Method" diff --git a/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr b/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr index 5c94503..8c44046 100644 --- a/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr +++ b/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr @@ -25,7 +25,7 @@ module AmberLSP::Rules::FSDD # def initialize with at least one typed parameter: matches ( ... : ... ) init_typed_re = /^\s*def\s+initialize\s*\([^)]*:[^)]*\)/ # Public def perform or call; private def perform|call is excluded - perform_re = /^\s*def\s+(perform|call)\b/ + perform_re = /^\s*def\s+(perform|call)\b/ private_perform_re = /^\s*private\s+def\s+(perform|call)\b/ has_typed_init = lines.any? { |line| init_typed_re.matches?(line) } @@ -41,7 +41,7 @@ module AmberLSP::Rules::FSDD class_name = match[1] start_char = (match.begin(1) || 0).to_i32 - end_char = (match.end(1) || line.size).to_i32 + end_char = (match.end(1) || line.size).to_i32 issues = [] of String issues << "missing initialize with typed parameters" unless has_typed_init diff --git a/src/amber_lsp/rules/fsdd/story_grammar_rule.cr b/src/amber_lsp/rules/fsdd/story_grammar_rule.cr index 67669de..6d74b09 100644 --- a/src/amber_lsp/rules/fsdd/story_grammar_rule.cr +++ b/src/amber_lsp/rules/fsdd/story_grammar_rule.cr @@ -25,16 +25,16 @@ module AmberLSP::Rules::FSDD diagnostics = [] of Diagnostic lines = content.lines - in_block = false - block_lines = [] of String - block_start = 0 + in_block = false + block_lines = [] of String + block_start = 0 lines.each_with_index do |line, i| is_comment = line.strip.starts_with?("#") if is_comment unless in_block - in_block = true + in_block = true block_start = i block_lines = [] of String end @@ -53,7 +53,7 @@ module AmberLSP::Rules::FSDD private def emit_story_diagnostics( block_lines : Array(String), block_start : Int32, - diagnostics : Array(Diagnostic) + diagnostics : Array(Diagnostic), ) : Nil block_text = block_lines.join(" ") return unless STORY_INITIATOR_RE.matches?(block_text) From 8c37b0d8188efa33cd9dc3434c4c29bf0910bf1e Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 20:11:53 -0400 Subject: [PATCH 15/69] feat(amber-lsp): activate analysis across the Amber stack --- .../analyze_file_with_coverage_spec.cr | 19 ++++++++ spec/amber_lsp/project_context_spec.cr | 48 +++++++++++++++++++ spec/amber_lsp/rule_packs_spec.cr | 21 ++++++++ src/amber_lsp/analyze_file_with_coverage.cr | 2 +- .../analyze_project_files_with_rule_packs.cr | 8 ---- .../print_detected_rule_pack_contexts.cr | 2 - src/amber_lsp/project_context.cr | 33 +++++++++++-- .../rules/fsdd/doc_block_required_rule.cr | 4 ++ .../rules/fsdd/method_type_signature_rule.cr | 4 ++ .../fsdd/process_manager_structure_rule.cr | 4 ++ .../rules/fsdd/story_grammar_rule.cr | 4 ++ 11 files changed, 134 insertions(+), 15 deletions(-) diff --git a/spec/amber_lsp/analyze_file_with_coverage_spec.cr b/spec/amber_lsp/analyze_file_with_coverage_spec.cr index 7282dfd..b92e7dc 100644 --- a/spec/amber_lsp/analyze_file_with_coverage_spec.cr +++ b/spec/amber_lsp/analyze_file_with_coverage_spec.cr @@ -1,5 +1,6 @@ require "./spec_helper" require "../../src/amber_lsp/rules/controllers/naming_rule" +require "../../src/amber_lsp/rules/fsdd/method_type_signature_rule" require "../../src/amber_lsp/analyze_file_with_coverage" describe AmberLSP::AnalyzeFileWithCoverage do @@ -32,6 +33,24 @@ describe AmberLSP::AnalyzeFileWithCoverage do end end + it "covers a Grant-only project and runs FSDD rules there" do + with_tempdir do |project| + Dir.mkdir_p(File.join(project, "src/models")) + File.write(File.join(project, "shard.yml"), "name: grant\nversion: 0.1.0\n") + file_path = File.join(project, "src/models/account.cr") + content = "def label\n \"account\"\nend\n" + File.write(file_path, content) + AmberLSP::Rules::RuleRegistry.register(AmberLSP::Rules::FSDD::MethodTypeSignatureRule.new) + + coverage = AmberLSP::AnalyzeFileWithCoverage.new(file_path).perform + + coverage.should be_a(AmberLSP::Coverage::Covered) + if covered = coverage.as?(AmberLSP::Coverage::Covered) + covered.list_of_diagnostics.map(&.code).should contain("fsdd/method-type-signature") + end + end + end + it "returns Declined with a reason for a project outside the stack" do with_tempdir do |project| File.write(File.join(project, "shard.yml"), "name: plain_crystal_app\nversion: 0.1.0\n") diff --git a/spec/amber_lsp/project_context_spec.cr b/spec/amber_lsp/project_context_spec.cr index 808a44b..730f968 100644 --- a/spec/amber_lsp/project_context_spec.cr +++ b/spec/amber_lsp/project_context_spec.cr @@ -59,12 +59,60 @@ describe AmberLSP::ProjectContext do end end + it "detects every Amber V2 stack shard name and the Amber CLI name" do + stack_shard_names = ["amber", "amber_router", "grant", "gemma", "asset_pipeline", "micrate", "quartz_mailer", "amber_cli"] + + stack_shard_names.each do |shard_name| + with_tempdir do |dir| + File.write(File.join(dir, "shard.yml"), "name: #{shard_name}\nversion: 0.1.0\n") + + project_context = AmberLSP::ProjectContext.detect(dir) + + project_context.stack_project?.should be_true + project_context.amber_project?.should eq(shard_name == "amber") + end + end + end + + it "detects each Amber V2 stack dependency key" do + stack_dependency_names = ["amber", "amber_router", "grant", "gemma", "asset_pipeline", "micrate", "quartz_mailer"] + + stack_dependency_names.each do |dependency_name| + with_tempdir do |dir| + shard_content = <<-YAML + name: my_app + version: 0.1.0 + dependencies: + #{dependency_name}: + github: example/#{dependency_name} + YAML + File.write(File.join(dir, "shard.yml"), shard_content) + + project_context = AmberLSP::ProjectContext.detect(dir) + + project_context.stack_project?.should be_true + project_context.amber_project?.should eq(dependency_name == "amber") + end + end + end + + it "declines a project with no stack name or dependency" do + with_tempdir do |dir| + File.write(File.join(dir, "shard.yml"), "name: my_app\nversion: 0.1.0\ndependencies:\n kemal: {}\n") + + project_context = AmberLSP::ProjectContext.detect(dir) + + project_context.stack_project?.should be_false + end + end + it "returns false for invalid YAML" do with_tempdir do |dir| File.write(File.join(dir, "shard.yml"), "{{invalid yaml") ctx = AmberLSP::ProjectContext.detect(dir) ctx.amber_project?.should be_false + ctx.failure_reason.should_not be_nil end end end diff --git a/spec/amber_lsp/rule_packs_spec.cr b/spec/amber_lsp/rule_packs_spec.cr index 6d4244e..c93efec 100644 --- a/spec/amber_lsp/rule_packs_spec.cr +++ b/spec/amber_lsp/rule_packs_spec.cr @@ -91,6 +91,27 @@ describe "AmberLSP Grant tenancy rule pack v2" do end end + it "loads and runs the Grant pack in the library's own repository" do + with_tempdir do |root| + install_tenancy_fixture_app(root, "row_app") + File.write(File.join(root, "shard.yml"), "name: grant\nversion: 0.1.0\n") + FileUtils.rm_rf(File.join(root, "lib")) + + own_repository_pack_path = File.join(root, ".amber-lsp", "packs", "tenancy.yml") + Dir.mkdir_p(File.dirname(own_repository_pack_path)) + File.write(own_repository_pack_path, File.read(GRANT_TENANCY_PACK_FIXTURE_PATH)) + + file_path = File.join(root, "src", "controllers", "invoices_controller.cr") + content = "Invoice.unscoped.all\n" + coverage = AmberLSP::AnalyzeFileWithCoverage.new(file_path, content).perform + + coverage.should be_a(AmberLSP::Coverage::Covered) + if covered = coverage.as?(AmberLSP::Coverage::Covered) + covered.list_of_diagnostics.map(&.code).should contain("grant/chained-unscoped-in-request-code") + end + end + end + it "skips malformed and unreadable pack files" do with_tempdir do |root| pack_directory = File.join(root, "lib", "grant", ".amber-lsp", "packs") diff --git a/src/amber_lsp/analyze_file_with_coverage.cr b/src/amber_lsp/analyze_file_with_coverage.cr index fdc68e4..229b5af 100644 --- a/src/amber_lsp/analyze_file_with_coverage.cr +++ b/src/amber_lsp/analyze_file_with_coverage.cr @@ -22,7 +22,7 @@ module AmberLSP return Coverage::Failed.new(error) end - unless current_project_context.amber_project? + unless current_project_context.stack_project? return Coverage::Declined.new("project is not an Amber V2 stack project") end diff --git a/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr b/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr index 109a347..e54e1ed 100644 --- a/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr +++ b/src/amber_lsp/library_rule_packs/analyze_project_files_with_rule_packs.cr @@ -9,8 +9,6 @@ module AmberLSP::LibraryRulePacks def has_applicable_pack?(file_path : String, content : String) : Bool @list_of_rule_packs.any? do |rule_pack| - next true if library_is_project_root?(rule_pack) - DetermineProjectRulePackState.new(@project_context, rule_pack, file_path, content) .has_any_applicable_mode? end @@ -21,8 +19,6 @@ module AmberLSP::LibraryRulePacks list_of_diagnostics = [] of Rules::Diagnostic @list_of_rule_packs.each do |rule_pack| - next if library_is_project_root?(rule_pack) - project_state = DetermineProjectRulePackState.new( @project_context, rule_pack, @@ -52,10 +48,6 @@ module AmberLSP::LibraryRulePacks apply_configured_severity(list_of_diagnostics) end - private def library_is_project_root?(rule_pack : DescribeLibraryRulePack) : Bool - @project_context.shard_name == rule_pack.library_shard_name - end - private def rule_applies_to_file?(rule : DescribeLibraryRulePack::Rule, relative_file_path : String) : Bool applies = rule.list_of_applicable_globs.any? do |pattern| Rules::RuleRegistry.file_matches_pattern?(relative_file_path, pattern) diff --git a/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr b/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr index f64c0bf..dd4d7f0 100644 --- a/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr +++ b/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr @@ -14,8 +14,6 @@ module AmberLSP::LibraryRulePacks list_of_output_lines = [] of String list_of_rule_packs.each do |rule_pack| - next if project_context.shard_name == rule_pack.library_shard_name - project_state = DetermineProjectRulePackState.new(project_context, rule_pack, "", "") next unless project_state.has_any_applicable_mode? diff --git a/src/amber_lsp/project_context.cr b/src/amber_lsp/project_context.cr index f7eebc7..4112844 100644 --- a/src/amber_lsp/project_context.cr +++ b/src/amber_lsp/project_context.cr @@ -2,8 +2,19 @@ require "yaml" module AmberLSP class ProjectContext + STACK_SHARD_NAMES = [ + "amber", + "amber_router", + "grant", + "gemma", + "asset_pipeline", + "micrate", + "quartz_mailer", + ] + getter root_path : String getter? amber_project : Bool + getter? stack_project : Bool getter shard_name : String? getter failure_reason : String? @@ -12,6 +23,7 @@ module AmberLSP @amber_project : Bool = false, @shard_name : String? = nil, @failure_reason : String? = nil, + @stack_project : Bool = false, ) end @@ -23,13 +35,17 @@ module AmberLSP end shard_configuration = YAML.parse(File.read(shard_path)) - is_amber = has_amber_dependency?(shard_configuration) shard_name = shard_configuration["name"]?.try(&.as_s?) + list_of_dependency_names = dependency_names(shard_configuration) + is_amber = shard_name == "amber" || list_of_dependency_names.includes?("amber") + is_stack = stack_project_named?(shard_name) || + list_of_dependency_names.any? { |dependency_name| STACK_SHARD_NAMES.includes?(dependency_name) } ProjectContext.new( root_path, amber_project: is_amber, shard_name: shard_name, + stack_project: is_stack, ) rescue ex : YAML::ParseException ProjectContext.new(root_path, failure_reason: "shard.yml is invalid: #{ex.message}") @@ -54,11 +70,20 @@ module AmberLSP end end - private def self.has_amber_dependency?(shard_configuration : YAML::Any) : Bool + private def self.stack_project_named?(shard_name : String?) : Bool + return false unless shard_name + + STACK_SHARD_NAMES.includes?(shard_name) || shard_name == "amber_cli" + end + + private def self.dependency_names(shard_configuration : YAML::Any) : Array(String) dependencies = shard_configuration["dependencies"]? - return false unless dependencies + return [] of String unless dependencies + + dependency_hash = dependencies.as_h? + return [] of String unless dependency_hash - dependencies["amber"]? != nil + dependency_hash.keys.compact_map(&.as_s?) end end end diff --git a/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr b/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr index 72bb190..336c963 100644 --- a/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr +++ b/src/amber_lsp/rules/fsdd/doc_block_required_rule.cr @@ -21,6 +21,10 @@ module AmberLSP::Rules::FSDD ["src/**"] end + def requires_amber_project? : Bool + false + end + def check(file_path : String, content : String) : Array(Diagnostic) diagnostics = [] of Diagnostic lines = content.lines diff --git a/src/amber_lsp/rules/fsdd/method_type_signature_rule.cr b/src/amber_lsp/rules/fsdd/method_type_signature_rule.cr index 6932486..d2f4ecb 100644 --- a/src/amber_lsp/rules/fsdd/method_type_signature_rule.cr +++ b/src/amber_lsp/rules/fsdd/method_type_signature_rule.cr @@ -19,6 +19,10 @@ module AmberLSP::Rules::FSDD ["src/**"] end + def requires_amber_project? : Bool + false + end + def check(file_path : String, content : String) : Array(Diagnostic) diagnostics = [] of Diagnostic diff --git a/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr b/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr index 8c44046..9fae39b 100644 --- a/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr +++ b/src/amber_lsp/rules/fsdd/process_manager_structure_rule.cr @@ -16,6 +16,10 @@ module AmberLSP::Rules::FSDD ["src/**/process_managers/**", "src/**/processes/**"] end + def requires_amber_project? : Bool + false + end + def check(file_path : String, content : String) : Array(Diagnostic) return [] of Diagnostic unless file_path.includes?("process_managers/") || file_path.includes?("processes/") diff --git a/src/amber_lsp/rules/fsdd/story_grammar_rule.cr b/src/amber_lsp/rules/fsdd/story_grammar_rule.cr index 6d74b09..8589a8c 100644 --- a/src/amber_lsp/rules/fsdd/story_grammar_rule.cr +++ b/src/amber_lsp/rules/fsdd/story_grammar_rule.cr @@ -21,6 +21,10 @@ module AmberLSP::Rules::FSDD ["src/**"] end + def requires_amber_project? : Bool + false + end + def check(file_path : String, content : String) : Array(Diagnostic) diagnostics = [] of Diagnostic lines = content.lines From 107b96ead6affdf88a439dfe5908009875ebec24 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 20:16:15 -0400 Subject: [PATCH 16/69] fix(amber-lsp): scope action checks to public methods --- .../controllers/action_return_rule_spec.cr | 84 ++++++++++++++++++- .../rules/controllers/action_return_rule.cr | 40 ++++++--- 2 files changed, 111 insertions(+), 13 deletions(-) diff --git a/spec/amber_lsp/rules/controllers/action_return_rule_spec.cr b/spec/amber_lsp/rules/controllers/action_return_rule_spec.cr index 76ff0ee..482405e 100644 --- a/spec/amber_lsp/rules/controllers/action_return_rule_spec.cr +++ b/spec/amber_lsp/rules/controllers/action_return_rule_spec.cr @@ -162,7 +162,7 @@ describe AmberLSP::Rules::Controllers::ActionReturnRule do render("index.ecr") end - private def helper_method + private def get_current_user "helper" end end @@ -182,7 +182,7 @@ describe AmberLSP::Rules::Controllers::ActionReturnRule do private - def helper_method + def logged_in? "helper" end end @@ -193,6 +193,86 @@ describe AmberLSP::Rules::Controllers::ActionReturnRule do diagnostics.should be_empty end + it "skips methods after protected keyword" do + content = <<-CRYSTAL + class HomeController < ApplicationController + def index + render("index.ecr") + end + + protected + + def establish_session + "helper" + end + end + CRYSTAL + + rule = AmberLSP::Rules::Controllers::ActionReturnRule.new + diagnostics = rule.check("src/controllers/home_controller.cr", content) + diagnostics.should be_empty + end + + it "skips methods declared with protected def" do + content = <<-CRYSTAL + class HomeController < ApplicationController + def index + render("index.ecr") + end + + protected def begin_mfa_challenge + "helper" + end + end + CRYSTAL + + rule = AmberLSP::Rules::Controllers::ActionReturnRule.new + diagnostics = rule.check("src/controllers/home_controller.cr", content) + diagnostics.should be_empty + end + + it "treats methods after public keyword as actions" do + content = <<-CRYSTAL + class HomeController < ApplicationController + protected + + public + + def index + @users = User.all + end + end + CRYSTAL + + rule = AmberLSP::Rules::Controllers::ActionReturnRule.new + diagnostics = rule.check("src/controllers/home_controller.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("index") + end + + it "resets visibility when the next controller class starts" do + content = <<-CRYSTAL + class AuthController < ApplicationController + private + + def session_helper + "helper" + end + end + + class UsersController < ApplicationController + def index + @users = User.all + end + end + CRYSTAL + + rule = AmberLSP::Rules::Controllers::ActionReturnRule.new + diagnostics = rule.check("src/controllers/users_controller.cr", content) + diagnostics.size.should eq(1) + diagnostics[0].message.should contain("index") + end + it "skips files not in controllers/ directory" do content = <<-CRYSTAL class SomeService diff --git a/src/amber_lsp/rules/controllers/action_return_rule.cr b/src/amber_lsp/rules/controllers/action_return_rule.cr index 00d638c..8ac5681 100644 --- a/src/amber_lsp/rules/controllers/action_return_rule.cr +++ b/src/amber_lsp/rules/controllers/action_return_rule.cr @@ -1,9 +1,12 @@ module AmberLSP::Rules::Controllers class ActionReturnRule < AmberLSP::Rules::BaseRule - RESPONSE_METHODS = ["render", "redirect_to", "redirect_back", "respond_with", "halt!"] - SKIPPED_METHODS = ["initialize", "before_action", "after_action", "before_filter", "after_filter"] - VISIBILITY_CHANGE = /^\s*(private|protected)\s*$/ - CLASS_METHOD = /^\s{2,4}def\s+self\./ + RESPONSE_METHODS = ["render", "redirect_to", "redirect_back", "respond_with", "halt!"] + SKIPPED_METHODS = ["initialize", "before_action", "after_action", "before_filter", "after_filter"] + VISIBILITY_CHANGE = /^\s*(private|protected|public)\s*$/ + METHOD_DECLARATION = /^(\s{2,4})(?:(private|protected|public)\s+)?def\s+([\w!?]+)/ + CLASS_METHOD = /^\s{2,4}(?:(?:private|protected|public)\s+)?def\s+self\./ + CLASS_DECLARATION = /^(\s*)class\b/ + END_DECLARATION = /^(\s*)end\b/ def id : String "amber/action-return-type" @@ -36,32 +39,47 @@ module AmberLSP::Rules::Controllers method_indent = 0 has_response_call = false is_private_section = false + list_of_class_indents = [] of Int32 lines.each_with_index do |line, line_number| + if class_match = CLASS_DECLARATION.match(line) + list_of_class_indents << class_match[1].size + is_private_section = false + end + # Track visibility section changes - if VISIBILITY_CHANGE.matches?(line) - is_private_section = true + if visibility_match = VISIBILITY_CHANGE.match(line) + is_private_section = visibility_match[1] != "public" next end + if end_match = END_DECLARATION.match(line) + if list_of_class_indents.last? == end_match[1].size + list_of_class_indents.pop + is_private_section = false + end + end + # Detect method start at standard 2-space indent (methods inside a class) - method_match = /^(\s{2,4})def\s+(\w+)/.match(line) + method_match = METHOD_DECLARATION.match(line) if method_match && !in_public_method next if CLASS_METHOD.matches?(line) indent = method_match[1].size - name = method_match[2] + visibility = method_match[2]? + name = method_match[3] # Skip private/protected methods and special methods + is_private_section = false if visibility == "public" + next if visibility == "private" || visibility == "protected" next if is_private_section next if SKIPPED_METHODS.includes?(name) - next if line.includes?("private def") || line.includes?("protected def") in_public_method = true method_name = name method_line = line_number - method_start_char = (method_match.begin(2) || 0).to_i32 - method_end_char = (method_match.end(2) || line.size).to_i32 + method_start_char = (method_match.begin(3) || 0).to_i32 + method_end_char = (method_match.end(3) || line.size).to_i32 method_indent = indent has_response_call = false next From 84f434b693f776b065a66c0a364a0a12110da826 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 20:18:53 -0400 Subject: [PATCH 17/69] feat(amber-lsp): exercise unsaved didChange in smoke probe --- scripts/lsp_smoke.rb | 69 ++++++++++++++++++++++++++++++++------------ 1 file changed, 51 insertions(+), 18 deletions(-) diff --git a/scripts/lsp_smoke.rb b/scripts/lsp_smoke.rb index 559fd4e..e20978a 100755 --- a/scripts/lsp_smoke.rb +++ b/scripts/lsp_smoke.rb @@ -15,10 +15,11 @@ # scripts/lsp_smoke.rb --server /path/to/amber-lsp # scripts/lsp_smoke.rb --timeout 20 --keep # keep the fixture project # -# Exit 0 = the violating fixture produced >= 1 diagnostic AND the clean fixture -# produced exactly 0. Exit 1 = live-fire expectations not met. Exit 2 = could -# not run at all (no binary, handshake failure, timeout). A timeout is never -# reported as a pass — "I could not measure it" is not "it is clean". +# Exit 0 = the violating fixture produced >= 1 diagnostic, the clean fixture +# produced exactly 0, and an unsaved didChange produced its expected diagnostic. +# Exit 1 = live-fire expectations not met. Exit 2 = could not run at all (no +# binary, handshake failure, timeout). A timeout is never reported as a pass — +# "I could not measure it" is not "it is clean". # # Ruby 2.6 compatible on purpose: it must run under macOS system ruby with no # gems, so it works anywhere the binary does. @@ -92,10 +93,9 @@ def frame(message) "Content-Length: #{json.bytesize}\r\n\r\n#{json}" end - # A minimal project the LSP will actually accept: ProjectContext.detect only - # switches diagnostics on when shard.yml has an `amber` DEPENDENCY. Without - # it the server stays silent and every file looks clean — the exact false - # green this script exists to make impossible. + # A minimal project the LSP will actually accept: shard.yml names an Amber + # stack shard. Without stack coverage the server stays silent and every file + # looks clean — the exact false green this script exists to make impossible. def build_fixture(dir) FileUtils.mkdir_p(File.join(dir, "src", "controllers")) FileUtils.mkdir_p(File.join(dir, "spec", "controllers")) @@ -138,6 +138,10 @@ def session(server_bin, dir, timeout) root_uri = "file://#{dir}" bad_uri = "file://#{dir}/src/controllers/users_controller.cr" good_uri = "file://#{dir}/src/controllers/posts_controller.cr" + good_file_path = File.join(dir, "src", "controllers", "posts_controller.cr") + clean_content = File.read(good_file_path) + unsaved_content = clean_content.sub("PostsController", "PostsHandler") + raise "could not prepare unsaved didChange content" if unsaved_content == clean_content messages = [ frame("jsonrpc" => "2.0", "id" => 1, "method" => "initialize", @@ -151,13 +155,17 @@ def session(server_bin, dir, timeout) frame("jsonrpc" => "2.0", "method" => "textDocument/didOpen", "params" => { "textDocument" => { "uri" => good_uri, "languageId" => "crystal", "version" => 1, - "text" => File.read(File.join(dir, "src", "controllers", "posts_controller.cr")) + "text" => clean_content } }), + frame("jsonrpc" => "2.0", "method" => "textDocument/didChange", + "params" => { "textDocument" => { "uri" => good_uri, "version" => 2 }, + "contentChanges" => [{ "text" => unsaved_content }] }), frame("jsonrpc" => "2.0", "id" => 2, "method" => "shutdown"), frame("jsonrpc" => "2.0", "method" => "exit") ] - published = {} + published = Hash.new { |hash, uri| hash[uri] = [] } + published_count = 0 initialized = false io = IO.popen([server_bin], "r+", err: File::NULL) @@ -169,7 +177,7 @@ def session(server_bin, dir, timeout) reader = FrameReader.new(io) deadline = Time.now + timeout - while published.size < 2 || !initialized + while published_count < 3 || !initialized raw = reader.read_frame(deadline) break if raw.nil? @@ -177,7 +185,8 @@ def session(server_bin, dir, timeout) initialized = true if msg["id"] == 1 && msg.key?("result") next unless msg["method"] == "textDocument/publishDiagnostics" - published[msg["params"]["uri"]] = msg + published[msg["params"]["uri"]] << msg + published_count += 1 end ensure begin @@ -187,7 +196,14 @@ def session(server_bin, dir, timeout) end end - { initialized: initialized, published: published, bad_uri: bad_uri, good_uri: good_uri } + { + initialized: initialized, + published: published, + bad_uri: bad_uri, + good_uri: good_uri, + good_file_path: good_file_path, + disk_content_unchanged: File.read(good_file_path) == clean_content + } end def describe(diagnostics) @@ -219,7 +235,7 @@ def main(argv) unless File.file?(server) && File.executable?(server) warn "lsp_smoke: no executable server at #{server}" - warn "lsp_smoke: build it first — CRYSTAL=crystal-alpha shards build amber-lsp" + warn "lsp_smoke: build it first — CRYSTAL=crystal-alpha shards-alpha build amber-lsp --release --no-debug" return EXIT_CANNOT end @@ -237,13 +253,17 @@ def main(argv) return EXIT_CANNOT end - bad = result[:published][result[:bad_uri]] - good = result[:published][result[:good_uri]] + bad_notifications = result[:published][result[:bad_uri]] || [] + good_notifications = result[:published][result[:good_uri]] || [] + bad = bad_notifications[0] + good = good_notifications[0] + changed = good_notifications[1] - if bad.nil? || good.nil? + if bad.nil? || good.nil? || changed.nil? missing = [] missing << "violating fixture" if bad.nil? missing << "clean fixture" if good.nil? + missing << "didChange fixture" if changed.nil? warn "lsp_smoke: no publishDiagnostics for #{missing.join(' and ')} within #{timeout}s." warn "lsp_smoke: a silent server is NOT a clean server — treating as could-not-measure." return EXIT_CANNOT @@ -251,6 +271,7 @@ def main(argv) bad_diags = bad["params"]["diagnostics"] good_diags = good["params"]["diagnostics"] + changed_diags = changed["params"]["diagnostics"] puts "--- publishDiagnostics: VIOLATING fixture (src/controllers/users_controller.cr) ---" puts JSON.pretty_generate(bad) @@ -260,6 +281,10 @@ def main(argv) puts JSON.pretty_generate(good) puts describe(good_diags) puts + puts "--- publishDiagnostics: UNSAVED didChange violation (posts_controller.cr) ---" + puts JSON.pretty_generate(changed) + puts describe(changed_diags) + puts ok = true if bad_diags.empty? @@ -270,9 +295,17 @@ def main(argv) warn "FAIL: clean fixture produced #{good_diags.size} diagnostic(s) (expected 0)." ok = false end + unless changed_diags.any? { |diagnostic| diagnostic["code"] == "amber/controller-naming" } + warn "FAIL: unsaved didChange content did not produce amber/controller-naming." + ok = false + end + unless result[:disk_content_unchanged] + warn "FAIL: didChange altered the clean fixture on disk." + ok = false + end if ok - puts "LIVE-FIRE OK: violating=#{bad_diags.size} diagnostic(s), clean=0." + puts "LIVE-FIRE OK: violating=#{bad_diags.size} diagnostic(s), clean=0, didChange=#{changed_diags.size} unsaved diagnostic(s), disk=unchanged." EXIT_OK else EXIT_FAILED From 44b142f877525fde4ec22df0466dc9145bdefaab Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 20:46:35 -0400 Subject: [PATCH 18/69] feat(amber-cli): record agent setup metadata --- spec/commands/setup_agent_spec.cr | 4 ++++ src/amber_cli/commands/setup_agent.cr | 31 ++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr index ef35252..c21c632 100644 --- a/spec/commands/setup_agent_spec.cr +++ b/spec/commands/setup_agent_spec.cr @@ -82,6 +82,10 @@ describe "amber setup:agent" do first_instructions.should contain("# Existing Claude instructions") first_instructions.should contain("crystal-alpha spec --affected") first_script.should contain("build --no-codegen 'src/custom_entry.cr'") + setup_manifest = AmberCLI::Agent::AgentSetupManifest.from_json(File.read(".amber/agent_setup.json")) + setup_manifest.amber_cli_version.should eq(AmberCli::VERSION) + setup_manifest.minimum_amber_lsp_version.should eq("1.0.0") + setup_manifest.generated_hook_version.should eq("2") File.file?(".lsp.json").should be_true File.info("bin/amber-agent-hook").permissions.to_i.&(0o111).should_not eq(0) command.list_of_info_messages.count("Updated: bin/amber-agent-hook").should eq(1) diff --git a/src/amber_cli/commands/setup_agent.cr b/src/amber_cli/commands/setup_agent.cr index 96b0800..3286006 100644 --- a/src/amber_cli/commands/setup_agent.cr +++ b/src/amber_cli/commands/setup_agent.cr @@ -1,9 +1,25 @@ require "json" require "yaml" require "../core/base_command" +require "../../version" require "./setup_lsp" module AmberCLI::Agent + class AgentSetupManifest + include JSON::Serializable + + property amber_cli_version : String + property minimum_amber_lsp_version : String + property generated_hook_version : String + + def initialize( + @amber_cli_version : String, + @minimum_amber_lsp_version : String, + @generated_hook_version : String, + ) + end + end + class HookHandler include JSON::Serializable include JSON::Serializable::Unmapped @@ -84,6 +100,8 @@ module AmberCLI::Commands # Installs the optional Claude Code and Codex feedback loop into an Amber V2 app. class SetupAgentCommand < AmberCLI::Core::BaseCommand AGENT_HOOK_SCRIPT = {{ read_file("#{__DIR__}/../templates/agent/amber-agent-hook") }} + MINIMUM_AMBER_LSP_VERSION = "1.0.0" + GENERATED_HOOK_VERSION = "2" DOCUMENT_START = "" DOCUMENT_END = "" @@ -101,6 +119,7 @@ module AmberCLI::Commands write_merged_hooks(path) end write_hook_script(main_file) + write_setup_manifest ["CLAUDE.md", "AGENTS.md"].each do |path| append_agent_loop_instructions(path) end @@ -141,7 +160,7 @@ module AmberCLI::Commands private def write_hook_script(main_file : String) : Nil path = "bin/amber-agent-hook" - content = AGENT_HOOK_SCRIPT.sub("__AMBER_MAIN__", main_file) + content = AGENT_HOOK_SCRIPT.gsub("__AMBER_MAIN__", main_file) Dir.mkdir_p("bin") if !File.file?(path) || File.read(path) != content File.write(path, content) @@ -150,6 +169,16 @@ module AmberCLI::Commands File.chmod(path, 0o755) end + private def write_setup_manifest : Nil + manifest = AmberCLI::Agent::AgentSetupManifest.new( + AmberCli::VERSION, + MINIMUM_AMBER_LSP_VERSION, + GENERATED_HOOK_VERSION, + ) + Dir.mkdir_p(".amber") + File.write(".amber/agent_setup.json", manifest.to_pretty_json + "\n") + end + private def append_agent_loop_instructions(path : String) : Nil content = File.file?(path) ? File.read(path) : "" if content.includes?(DOCUMENT_START) && content.includes?(DOCUMENT_END) From 1283a1266e63c49c6f18505c1c9f9745bb3deab0 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 20:56:38 -0400 Subject: [PATCH 19/69] feat(lsp): build layered Crystal docs API index --- .../lookup/build_layered_api_index_spec.cr | 66 ++++ spec/amber_lsp/lookup/index_cache_spec.cr | 94 +++++ .../lookup/normalize_crystal_docs_spec.cr | 50 +++ .../api_lookup/crystal_docs_small.json | 67 ++++ src/amber_lsp.cr | 1 + .../lookup/build_crystal_docs_json.cr | 54 +++ .../lookup/build_layered_api_index.cr | 347 ++++++++++++++++++ src/amber_lsp/lookup/index_cache.cr | 184 ++++++++++ src/amber_lsp/lookup/index_models.cr | 187 ++++++++++ .../lookup/normalize_crystal_docs.cr | 111 ++++++ src/amber_lsp/lookup/source_models.cr | 47 +++ 11 files changed, 1208 insertions(+) create mode 100644 spec/amber_lsp/lookup/build_layered_api_index_spec.cr create mode 100644 spec/amber_lsp/lookup/index_cache_spec.cr create mode 100644 spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr create mode 100644 spec/fixtures/api_lookup/crystal_docs_small.json create mode 100644 src/amber_lsp/lookup/build_crystal_docs_json.cr create mode 100644 src/amber_lsp/lookup/build_layered_api_index.cr create mode 100644 src/amber_lsp/lookup/index_cache.cr create mode 100644 src/amber_lsp/lookup/index_models.cr create mode 100644 src/amber_lsp/lookup/normalize_crystal_docs.cr create mode 100644 src/amber_lsp/lookup/source_models.cr diff --git a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr new file mode 100644 index 0000000..2bf5c1d --- /dev/null +++ b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr @@ -0,0 +1,66 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/build_layered_api_index" + +describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do + it "builds project and standard library layers, then reuses their fresh cache" do + with_tempdir do |root| + project_root = File.join(root, "project") + cache_root = File.join(root, "cache") + Dir.mkdir_p(File.join(project_root, "src")) + Dir.mkdir_p(File.join(project_root, "lib", "tiny", "src")) + File.write( + File.join(project_root, "shard.yml"), + "name: lookup_fixture\nversion: 1.0.0\ndependencies:\n tiny:\n github: example/tiny\ntargets:\n app:\n main: src/lookup_fixture.cr\n", + ) + File.write( + File.join(project_root, "shard.lock"), + "version: 2.0\nshards:\n tiny:\n git: https://example.test/tiny.git\n version: 1.2.3+git.commit.abcd\n", + ) + File.write( + File.join(project_root, "lib", "tiny", "shard.yml"), + "name: tiny\nversion: 1.2.3\n", + ) + File.write( + File.join(project_root, "lib", "tiny", "src", "tiny.cr"), + "module Tiny\n class Client\n def ping : Bool\n true\n end\n end\nend\n", + ) + File.write( + File.join(project_root, "src", "lookup_fixture.cr"), + "require \"tiny\"\nclass FixtureUser\n def id : Int64?\n 1_i64\n end\nend\n", + ) + + builder = AmberLSP::Lookup::BuildLayeredAPIIndex.new(project_root, cache_root) + first_build = builder.perform + second_build = builder.perform + + project_layer = first_build.find { |cached| cached.layer.try(&.layer_kind) == "project" } + library_layer = first_build.find { |cached| cached.layer.try(&.layer_kind) == "library" } + standard_library_layer = first_build.find { |cached| cached.layer.try(&.layer_kind) == "stdlib" } + project_layer.should_not be_nil + library_layer.should_not be_nil + standard_library_layer.should_not be_nil + project_layer.not_nil!.freshness.should eq("fresh") + library_layer.not_nil!.freshness.should eq("fresh") + standard_library_layer.not_nil!.freshness.should eq("fresh") + + fixture_user = project_layer.not_nil!.layer.not_nil!.list_of_types.find { |type| type.name == "FixtureUser" } + fixture_user.should_not be_nil + fixture_user.not_nil!.list_of_instance_methods.first.return_type.should eq("Int64 | ::Nil") + + tiny_client = library_layer.not_nil!.layer.not_nil!.list_of_types.find { |type| type.name == "Tiny::Client" } + tiny_client.should_not be_nil + tiny_client.not_nil!.list_of_instance_methods.first.name.should eq("ping") + + standard_types = standard_library_layer.not_nil!.layer.not_nil!.list_of_types + dir_type = standard_types.find { |type| type.name == "Dir" } + file_type = standard_types.find { |type| type.name == "File" } + dir_type.should_not be_nil + file_type.should_not be_nil + dir_type.not_nil!.list_of_class_methods.any? { |method| method.name == "mkdir_p" }.should be_true + file_type.not_nil!.list_of_class_methods.any? { |method| method.name == "mkdir_p" }.should be_false + + second_build.size.should eq(3) + second_build.each { |cached| cached.freshness.should eq("fresh") } + end + end +end diff --git a/spec/amber_lsp/lookup/index_cache_spec.cr b/spec/amber_lsp/lookup/index_cache_spec.cr new file mode 100644 index 0000000..5c22069 --- /dev/null +++ b/spec/amber_lsp/lookup/index_cache_spec.cr @@ -0,0 +1,94 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/index_cache" + +describe "AmberLSP::Lookup::CalculateProjectLayerKey#perform" do + it "changes when indexed source, shard metadata, or docs flags change" do + with_tempdir do |root| + Dir.mkdir_p(File.join(root, "src")) + Dir.mkdir_p(File.join(root, "config")) + File.write(File.join(root, "shard.yml"), "name: sample\nversion: 1.0.0\n") + File.write(File.join(root, "src", "sample.cr"), "class Sample\nend\n") + File.write(File.join(root, "config", "app.cr"), "APP_NAME = \"sample\"\n") + + original_key = AmberLSP::Lookup::CalculateProjectLayerKey.new(root, [] of String, "Crystal 1.21.0").perform + File.write(File.join(root, "spec.cr"), "class Unindexed\nend\n") + key_ignoring_unindexed_file = AmberLSP::Lookup::CalculateProjectLayerKey.new(root, [] of String, "Crystal 1.21.0").perform + key_with_docs_flag = AmberLSP::Lookup::CalculateProjectLayerKey.new(root, ["grant_docs"], "Crystal 1.21.0").perform + + key_ignoring_unindexed_file.should eq(original_key) + key_with_docs_flag.should_not eq(original_key) + + File.write(File.join(root, "src", "sample.cr"), "class Sample\n def id : Int32\n 1\n end\nend\n") + key_with_changed_source = AmberLSP::Lookup::CalculateProjectLayerKey.new(root, [] of String, "Crystal 1.21.0").perform + key_with_changed_source.should_not eq(original_key) + + File.write(File.join(root, "config", "app.cr"), "APP_NAME = \"changed\"\n") + key_with_changed_config = AmberLSP::Lookup::CalculateProjectLayerKey.new(root, [] of String, "Crystal 1.21.0").perform + key_with_changed_config.should_not eq(original_key) + + File.write(File.join(root, "shard.yml"), "name: sample\nversion: 1.0.1\n") + key_with_changed_manifest = AmberLSP::Lookup::CalculateProjectLayerKey.new(root, [] of String, "Crystal 1.21.0").perform + key_with_changed_manifest.should_not eq(original_key) + end + end +end + +describe "AmberLSP::Lookup::CalculateStandardLibraryLayerKey#perform" do + it "pins the standard library layer to the compiler identity and docs flags" do + first_key = AmberLSP::Lookup::CalculateStandardLibraryLayerKey.new("Crystal 1.21.0").perform + changed_compiler = AmberLSP::Lookup::CalculateStandardLibraryLayerKey.new("Crystal 1.21.1").perform + changed_flags = AmberLSP::Lookup::CalculateStandardLibraryLayerKey.new("Crystal 1.21.0", ["stdlib_docs"]).perform + + changed_compiler.should_not eq(first_key) + changed_flags.should_not eq(first_key) + end +end + +describe "AmberLSP::Lookup::CalculateLibraryLayerKey#perform" do + it "includes the shard identity, locked version, docs flags, and compiler version" do + first_key = AmberLSP::Lookup::CalculateLibraryLayerKey.new("grant", "0.23.4+git.commit.abc123", ["grant_docs"], "Crystal 1.21.0").perform + same_key = AmberLSP::Lookup::CalculateLibraryLayerKey.new("grant", "0.23.4+git.commit.abc123", ["grant_docs"], "Crystal 1.21.0").perform + changed_version = AmberLSP::Lookup::CalculateLibraryLayerKey.new("grant", "0.23.5", ["grant_docs"], "Crystal 1.21.0").perform + changed_flags = AmberLSP::Lookup::CalculateLibraryLayerKey.new("grant", "0.23.4+git.commit.abc123", [] of String, "Crystal 1.21.0").perform + changed_compiler = AmberLSP::Lookup::CalculateLibraryLayerKey.new("grant", "0.23.4+git.commit.abc123", ["grant_docs"], "Crystal 1.21.1").perform + + same_key.should eq(first_key) + changed_version.should_not eq(first_key) + changed_flags.should_not eq(first_key) + changed_compiler.should_not eq(first_key) + end +end + +describe "AmberLSP::Lookup::APIIndexCache#write_layer and #load_layer" do + it "atomically stores a fresh layer, returns stale cache on a changed key, and reports unavailable without cache" do + with_tempdir do |root| + cache_root = File.join(root, "cache") + project_root = File.join(root, "project") + layer = AmberLSP::Lookup::APIIndexLayer.new( + "project", + "sample", + "key-v1", + project_root, + [] of String, + [] of AmberLSP::Lookup::APIIndexType, + ) + cache = AmberLSP::Lookup::APIIndexCache.new(cache_root, project_root) + + cache.write_layer(layer) + fresh = cache.load_layer("project", "sample", "key-v1") + stale = cache.load_layer("project", "sample", "key-v2") + unavailable = AmberLSP::Lookup::APIIndexCache.new(File.join(root, "empty"), project_root) + .load_layer("project", "sample", "key-v1") + + fresh.freshness.should eq("fresh") + fresh.layer.should_not be_nil + fresh.layer.not_nil!.layer_key.should eq("key-v1") + stale.freshness.should eq("stale") + stale.layer.should_not be_nil + stale.layer.not_nil!.layer_key.should eq("key-v1") + unavailable.freshness.should eq("unavailable") + unavailable.layer.should be_nil + Dir.glob(File.join(cache_root, "**", "*.tmp-*")).should be_empty + end + end +end diff --git a/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr b/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr new file mode 100644 index 0000000..2bf9f14 --- /dev/null +++ b/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr @@ -0,0 +1,50 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/normalize_crystal_docs" + +describe "AmberLSP::Lookup::NormalizeCrystalDocs#perform" do + it "normalizes nested types, method overloads, module relations, and source locations" do + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_small.json") + normalizer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + "/project", + "project", + "project", + "project-key", + [] of String, + ) + + layer = normalizer.perform + user = layer.list_of_types.find { |type| type.name == "FixtureAPI::User" } + user.should_not be_nil + + user = user.not_nil! + user.kind.should eq("class") + user.list_of_ancestor_names.should eq(["Grant::Base"]) + user.list_of_included_module_names.should eq(["FixtureAPI::Timestamped"]) + user.list_of_extended_module_names.should eq(["FixtureAPI::QueryMethods"]) + user.location_path.should eq("/project/src/models/user.cr") + user.location_line.should eq(3) + + id_method = user.list_of_instance_methods.first + id_method.owner.should eq("FixtureAPI::User") + id_method.name.should eq("id") + id_method.args_string.should eq("()") + id_method.return_type.should eq("Int64 | ::Nil") + id_method.doc_line.should eq("The user's primary key.") + id_method.source_path.should eq("/project/src/models/user.cr") + id_method.source_line.should eq(12) + id_method.source_layer.should eq("project") + id_method.abstract?.should be_false + id_method.macro?.should be_false + + token_macro = user.list_of_macros.first + token_macro.name.should eq("has_token") + token_macro.macro?.should be_true + + query_methods = layer.list_of_types.find { |type| type.name == "FixtureAPI::QueryMethods" } + query_methods.should_not be_nil + where_overloads = query_methods.not_nil!.list_of_instance_methods.select { |method| method.name == "where" } + where_overloads.size.should eq(2) + where_overloads.map(&.abstract?).should eq([false, true]) + end +end diff --git a/spec/fixtures/api_lookup/crystal_docs_small.json b/spec/fixtures/api_lookup/crystal_docs_small.json new file mode 100644 index 0000000..0dacd7e --- /dev/null +++ b/spec/fixtures/api_lookup/crystal_docs_small.json @@ -0,0 +1,67 @@ +{ + "repository_name": "fixture_api", + "program": { + "kind": "module", + "name": "FixtureAPI", + "full_name": "FixtureAPI", + "abstract": false, + "types": [ + { + "kind": "class", + "name": "User", + "full_name": "FixtureAPI::User", + "abstract": false, + "locations": [{"filename": "src/models/user.cr", "line_number": 3}], + "ancestors": [{"kind": "class", "name": "Base", "full_name": "Grant::Base"}], + "included_modules": [{"kind": "module", "name": "Timestamped", "full_name": "FixtureAPI::Timestamped"}], + "extended_modules": [{"kind": "module", "name": "QueryMethods", "full_name": "FixtureAPI::QueryMethods"}], + "instance_methods": [ + { + "name": "id", + "abstract": false, + "args_string": "() : Int64 | ::Nil", + "doc": "The user's primary key.\nAdditional details.", + "location": {"filename": "src/models/user.cr", "line_number": 12, "url": null}, + "def": {"name": "id", "return_type": "Int64 | ::Nil"} + } + ], + "class_methods": [], + "macros": [ + { + "name": "has_token", + "abstract": false, + "args_string": "(name)", + "doc": "Adds a token column.", + "location": {"filename": "src/models/user.cr", "line_number": 8, "url": null}, + "def": {"name": "has_token"} + } + ] + }, + { + "kind": "module", + "name": "QueryMethods", + "full_name": "FixtureAPI::QueryMethods", + "abstract": false, + "locations": [{"filename": "src/query_methods.cr", "line_number": 1}], + "instance_methods": [ + { + "name": "where", + "abstract": false, + "args_string": "(field : String) : Query", + "doc": "Finds matching records.\nReturns a lazy query.", + "location": {"filename": "src/query_methods.cr", "line_number": 9, "url": null}, + "def": {"name": "where", "return_type": "Query"} + }, + { + "name": "where", + "abstract": true, + "args_string": "(fields : Hash(String, Int64)) : Query", + "doc": "Finds matching records by a map.", + "location": {"filename": "src/query_methods.cr", "line_number": 18, "url": null}, + "def": {"name": "where", "return_type": "Query"} + } + ] + } + ] + } +} diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 5631d47..5d18a5f 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -41,6 +41,7 @@ require "./amber_lsp/analyze_file_with_coverage" require "./amber_lsp/controller" require "./amber_lsp/server" require "./amber_lsp/check_file_for_diagnostics" +require "./amber_lsp/lookup/build_layered_api_index" if ARGV.includes?("--version") STDOUT.puts AmberLSP.version_line diff --git a/src/amber_lsp/lookup/build_crystal_docs_json.cr b/src/amber_lsp/lookup/build_crystal_docs_json.cr new file mode 100644 index 0000000..7dc6279 --- /dev/null +++ b/src/amber_lsp/lookup/build_crystal_docs_json.cr @@ -0,0 +1,54 @@ +require "process" + +require "./index_cache" + +module AmberLSP::Lookup + class BuildCrystalDocsJSON + def initialize( + @compiler_path : String, + @working_directory : String, + @entrypoint : String, + @project_name : String, + @project_version : String, + @docs_flags : Array(String), + @environment_overrides : Hash(String, String) = {} of String => String, + ) + end + + def perform : String + output = IO::Memory.new + error_output = IO::Memory.new + arguments = [ + "docs", + "--format=json", + "--project-name=#{@project_name}", + "--project-version=#{@project_version}", + ] + @docs_flags.each do |flag| + arguments << "-D" + arguments << flag + end + arguments << @entrypoint + + begin + status = Process.run( + @compiler_path, + arguments, + chdir: @working_directory, + env: @environment_overrides, + output: output, + error: error_output, + ) + rescue ex : IO::Error + raise APIIndexBuildError.new("Could not run crystal-alpha docs: #{ex.message}") + end + + unless status.success? + error_message = error_output.to_s.strip + raise APIIndexBuildError.new("crystal-alpha docs failed for #{@project_name}: #{error_message}") + end + + output.to_s + end + end +end diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr new file mode 100644 index 0000000..4c5e80a --- /dev/null +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -0,0 +1,347 @@ +require "digest/sha256" +require "process" +require "yaml" + +require "./build_crystal_docs_json" +require "./index_cache" +require "./normalize_crystal_docs" +require "./source_models" + +module AmberLSP::Lookup + struct CrystalAlphaIdentity + getter executable_path : String + getter version : String + getter llvm_version : String + + def initialize(@executable_path : String, @version : String, @llvm_version : String) + end + end + + class DetectCrystalAlpha + def initialize(@compiler_command : String = "crystal-alpha") + end + + def perform : CrystalAlphaIdentity + compiler_path = Process.find_executable(@compiler_command) + raise APIIndexBuildError.new("crystal-alpha is not available on PATH") unless compiler_path + + output = run_compiler_command(compiler_path, ["--version"]) + version = output.strip + llvm_version = version.lines.find(&.starts_with?("LLVM:")) + .try(&.split(':', 2).last?.try(&.strip)) || "unknown" + + CrystalAlphaIdentity.new(compiler_path, version, llvm_version) + end + + private def run_compiler_command(compiler_path : String, arguments : Array(String)) : String + output = IO::Memory.new + error_output = IO::Memory.new + status = Process.run(compiler_path, arguments, output: output, error: error_output) + unless status.success? + raise APIIndexBuildError.new("Could not inspect crystal-alpha: #{error_output.to_s.strip}") + end + output.to_s + rescue ex : IO::Error + raise APIIndexBuildError.new("Could not inspect crystal-alpha: #{ex.message}") + end + end + + class BuildLayeredAPIIndex + def initialize( + @root_path : String, + @cache_root : String = APIIndexCache.default_root, + @list_of_docs_flags_by_library : Hash(String, Array(String)) = {} of String => Array(String), + @compiler_command : String = "crystal-alpha", + ) + @root_path = File.expand_path(@root_path) + end + + def perform : Array(CachedAPIIndexLayer) + manifest = read_project_manifest + identity = DetectCrystalAlpha.new(@compiler_command).perform + cache = APIIndexCache.new(@cache_root, @root_path) + list_of_layers = [] of CachedAPIIndexLayer + project_name = manifest.name.empty? ? File.basename(@root_path) : manifest.name + project_flags = docs_flags_for(project_name) + project_key = CalculateProjectLayerKey.new(@root_path, project_flags, identity.version).perform + project_entrypoint = find_project_entrypoint(manifest) + + list_of_layers << build_or_load_layer( + cache, + identity, + "project", + project_name, + project_key, + @root_path, + project_flags, + project_entrypoint, + @root_path, + {} of String => String, + ) + + locked_shards = read_locked_shards + if !locked_shards.empty? + project_crystal_path = crystal_path_for_project_libraries(identity) + else + project_crystal_path = nil + end + + manifest.dependencies.keys.sort.each do |library_name| + next if locked_shards.has_key?(library_name) + + list_of_layers << CachedAPIIndexLayer.new( + nil, + "unavailable", + "#{library_name} is declared in shard.yml but is missing from shard.lock", + ) + end + + locked_shards.to_a.sort_by(&.first).each do |library_name, locked_shard| + library_root = File.join(@root_path, "lib", library_name) + unless File.directory?(library_root) + list_of_layers << CachedAPIIndexLayer.new(nil, "unavailable", "Locked shard #{library_name} is not installed under lib/") + next + end + + entrypoint = find_library_entrypoint(library_root, library_name) + unless entrypoint + list_of_layers << CachedAPIIndexLayer.new(nil, "unavailable", "Locked shard #{library_name} has no Crystal entrypoint under lib/#{library_name}/src") + next + end + + docs_flags = docs_flags_for(library_name) + layer_key = CalculateLibraryLayerKey.new(library_name, locked_shard.version, docs_flags, identity.version).perform + environment_overrides = {"CRYSTAL_PATH" => project_crystal_path || ""} + + list_of_layers << build_or_load_layer( + cache, + identity, + "library", + library_name, + layer_key, + library_root, + docs_flags, + entrypoint, + library_root, + environment_overrides, + ) + end + + list_of_layers << standard_library_layer(cache, identity) + + list_of_layers + end + + private def read_project_manifest : ProjectShardManifest + shard_path = File.join(@root_path, "shard.yml") + return ProjectShardManifest.new if !File.file?(shard_path) + + ProjectShardManifest.from_yaml(File.read(shard_path)) + rescue ex : YAML::ParseException | IO::Error + raise APIIndexBuildError.new("Could not read shard.yml: #{ex.message}") + end + + private def read_locked_shards : Hash(String, LockedShard) + lock_path = File.join(@root_path, "shard.lock") + return {} of String => LockedShard unless File.file?(lock_path) + + ShardLockfile.from_yaml(File.read(lock_path)).shards + rescue ex : YAML::ParseException | IO::Error + raise APIIndexBuildError.new("Could not read shard.lock: #{ex.message}") + end + + private def find_project_entrypoint(manifest : ProjectShardManifest) : String + target_entrypoints = manifest.targets.values.map(&.main).reject(&.empty?).sort + target_entrypoints.each do |entrypoint| + absolute_path = File.expand_path(entrypoint, @root_path) + return entrypoint if File.file?(absolute_path) + end + + if !manifest.name.empty? + conventional_entrypoint = File.join("src", "#{manifest.name.gsub('-', '_')}.cr") + return conventional_entrypoint if File.file?(File.join(@root_path, conventional_entrypoint)) + end + + preferred_entrypoints = ["config/application.cr", "src/application.cr"] + preferred_entrypoints.each do |entrypoint| + return entrypoint if File.file?(File.join(@root_path, entrypoint)) + end + + first_source = Dir.glob(File.join(@root_path, "src", "*.cr")).sort.first? + raise APIIndexBuildError.new("No Crystal entrypoint found under #{@root_path}/src") unless first_source + + first_source.sub(@root_path + "/", "") + end + + private def find_library_entrypoint(library_root : String, library_name : String) : String? + source_root = File.join(library_root, "src") + return nil unless File.directory?(source_root) + + conventional_name = library_name.gsub('-', '_') + conventional_entrypoint = File.join(source_root, "#{conventional_name}.cr") + return conventional_entrypoint if File.file?(conventional_entrypoint) + + Dir.glob(File.join(source_root, "*.cr")).sort.first? || Dir.glob(File.join(source_root, "**", "*.cr")).sort.first? + end + + private def docs_flags_for(library_name : String) : Array(String) + @list_of_docs_flags_by_library[library_name]?.try(&.sort) || [] of String + end + + private def build_or_load_layer( + cache : APIIndexCache, + identity : CrystalAlphaIdentity, + layer_kind : String, + layer_name : String, + layer_key : String, + source_root : String, + docs_flags : Array(String), + entrypoint : String, + working_directory : String, + environment_overrides : Hash(String, String), + ) : CachedAPIIndexLayer + cached_layer = cache.load_layer(layer_kind, layer_name, layer_key) + return cached_layer if cached_layer.freshness == "fresh" + + begin + docs_json = BuildCrystalDocsJSON.new( + identity.executable_path, + working_directory, + relative_entrypoint(entrypoint, working_directory), + layer_name, + "0", + docs_flags, + environment_overrides, + ).perform + layer = NormalizeCrystalDocs.new( + docs_json, + source_root, + layer_kind, + layer_name, + layer_key, + docs_flags, + ).perform + cache.write_layer(layer) + CachedAPIIndexLayer.new(layer, "fresh") + rescue ex : APIIndexBuildError | JSON::ParseException + return cached_layer if cached_layer.layer + + CachedAPIIndexLayer.new(nil, "unavailable", ex.message) + end + end + + private def standard_library_layer(cache : APIIndexCache, identity : CrystalAlphaIdentity) : CachedAPIIndexLayer + layer_key = CalculateStandardLibraryLayerKey.new(identity.version).perform + cached_layer = cache.load_layer("stdlib", "crystal", layer_key) + return cached_layer if cached_layer.freshness == "fresh" + + begin + source_root = crystal_source_root(identity) + docs_flags = [] of String + environment_overrides = {} of String => String + if llvm_config = find_llvm_config(identity.llvm_version) + environment_overrides["LLVM_CONFIG"] = llvm_config + end + + docs_json = BuildCrystalDocsJSON.new( + identity.executable_path, + File.dirname(source_root), + File.join("src", "docs_main.cr"), + "crystal", + "0", + docs_flags, + environment_overrides, + ).perform + layer = NormalizeCrystalDocs.new( + docs_json, + File.dirname(source_root), + "stdlib", + "crystal", + layer_key, + docs_flags, + ).perform + cache.write_layer(layer) + CachedAPIIndexLayer.new(layer, "fresh") + rescue ex : APIIndexBuildError | JSON::ParseException | IO::Error + return cached_layer if cached_layer.layer + + CachedAPIIndexLayer.new(nil, "unavailable", ex.message) + end + end + + private def crystal_source_root(identity : CrystalAlphaIdentity) : String + output = IO::Memory.new + error_output = IO::Memory.new + status = Process.run(identity.executable_path, ["env", "CRYSTAL_PATH"], output: output, error: error_output) + unless status.success? + raise APIIndexBuildError.new("Could not locate the Crystal standard library: #{error_output.to_s.strip}") + end + + source_root = output.to_s.strip.split(':').find do |path| + next false if path == "lib" + + File.file?(File.join(path, "docs_main.cr")) + end + raise APIIndexBuildError.new("Could not locate src/docs_main.cr from crystal-alpha") unless source_root + + source_root + rescue ex : IO::Error + raise APIIndexBuildError.new("Could not locate the Crystal standard library: #{ex.message}") + end + + private def crystal_path_for_project_libraries(identity : CrystalAlphaIdentity) : String + output = IO::Memory.new + error_output = IO::Memory.new + status = Process.run(identity.executable_path, ["env", "CRYSTAL_PATH"], output: output, error: error_output) + unless status.success? + raise APIIndexBuildError.new("Could not read crystal-alpha CRYSTAL_PATH: #{error_output.to_s.strip}") + end + "#{output.to_s.strip}:#{File.join(@root_path, "lib")}" + rescue ex : IO::Error + raise APIIndexBuildError.new("Could not read crystal-alpha CRYSTAL_PATH: #{ex.message}") + end + + private def find_llvm_config(llvm_version : String) : String? + version_parts = llvm_version.split('.') + major = version_parts.first? + minor = version_parts[1]? + return nil unless major && minor + + candidates = [] of String + candidates << "/opt/homebrew/opt/llvm@#{major}/bin/llvm-config" + candidates << "/usr/local/opt/llvm@#{major}/bin/llvm-config" + + if executable = Process.find_executable("llvm-config-#{major}.#{minor}") + candidates << executable + end + if executable = Process.find_executable("llvm-config-#{major}") + candidates << executable + end + if executable = Process.find_executable("llvm-config") + candidates << executable + end + + candidates.uniq.each do |candidate| + next unless File.file?(candidate) || Process.find_executable(candidate) + + version_output = IO::Memory.new + version_error = IO::Memory.new + status = Process.run(candidate, ["--version"], output: version_output, error: version_error) + next unless status.success? + next unless version_output.to_s.strip.starts_with?("#{major}.#{minor}") + + return candidate + rescue ex : IO::Error + next + end + + nil + end + + private def relative_entrypoint(entrypoint : String, working_directory : String) : String + return entrypoint unless Path[entrypoint].absolute? + + entrypoint.sub(File.expand_path(working_directory) + "/", "") + end + end +end diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr new file mode 100644 index 0000000..69e89a2 --- /dev/null +++ b/src/amber_lsp/lookup/index_cache.cr @@ -0,0 +1,184 @@ +require "digest/sha256" +require "json" +require "random/secure" + +require "./index_models" + +module AmberLSP::Lookup + class APIIndexBuildError < Exception + end + + class CalculateProjectLayerKey + def initialize(@root_path : String, @docs_flags : Array(String), @compiler_version : String) + end + + def perform : String + root_path = File.expand_path(@root_path) + list_of_index_paths = source_paths(root_path) + key_material = ["project", @compiler_version, @docs_flags.sort.join("\0")] + + list_of_index_paths.each do |path| + relative_path = path.sub(root_path + "/", "") + key_material << relative_path + key_material << Digest::SHA256.hexdigest(File.read(path)) + end + + key_material << "shard.yml" + shard_path = File.join(root_path, "shard.yml") + key_material << (File.file?(shard_path) ? Digest::SHA256.hexdigest(File.read(shard_path)) : "missing") + Digest::SHA256.hexdigest(key_material.join("\0")) + end + + private def source_paths(root_path : String) : Array(String) + list_of_paths = [] of String + {"src", "config"}.each do |directory_name| + directory_path = File.join(root_path, directory_name) + next unless File.directory?(directory_path) + + list_of_paths.concat(Dir.glob(File.join(directory_path, "**", "*.cr"))) + end + list_of_paths.sort + end + end + + class CalculateLibraryLayerKey + def initialize( + @library_name : String, + @locked_version_or_commit : String, + @docs_flags : Array(String), + @compiler_version : String, + ) + end + + def perform : String + key_material = [ + "library", + @library_name, + @locked_version_or_commit, + @docs_flags.sort.join("\0"), + @compiler_version, + ] + Digest::SHA256.hexdigest(key_material.join("\0")) + end + end + + class CalculateStandardLibraryLayerKey + def initialize(@compiler_version : String, @docs_flags : Array(String) = [] of String) + end + + def perform : String + key_material = ["stdlib", @compiler_version, @docs_flags.sort.join("\0")] + Digest::SHA256.hexdigest(key_material.join("\0")) + end + end + + struct CachedAPIIndexLayer + getter layer : APIIndexLayer? + getter freshness : String + getter failure_reason : String? + + def initialize(@layer : APIIndexLayer?, @freshness : String, @failure_reason : String? = nil) + end + end + + # :nodoc: + struct APIIndexCachePointer + include JSON::Serializable + + getter key : String + + def initialize(@key : String) + end + end + + class APIIndexCache + def initialize(@cache_root : String, @project_root_path : String) + end + + def self.default_root : String + File.join(Path.home.to_s, ".cache", "amber-lsp", "index") + end + + def write_layer(layer : APIIndexLayer) : String + directory_path = layer_directory(layer.layer_kind, layer.layer_name) + Dir.mkdir_p(directory_path) + + layer_path = File.join(directory_path, "#{layer.layer_key}.json") + write_atomically(layer_path, layer.to_json) + + latest_path = File.join(directory_path, "latest.json") + write_atomically(latest_path, APIIndexCachePointer.new(layer.layer_key).to_json) + layer_path + end + + def load_layer(layer_kind : String, layer_name : String, expected_key : String) : CachedAPIIndexLayer + directory_path = layer_directory(layer_kind, layer_name) + fresh_path = File.join(directory_path, "#{expected_key}.json") + + if File.file?(fresh_path) + if layer = load_matching_layer(fresh_path, layer_kind, layer_name, expected_key) + return CachedAPIIndexLayer.new(layer, "fresh") + end + end + + latest_path = File.join(directory_path, "latest.json") + return CachedAPIIndexLayer.new(nil, "unavailable") unless File.file?(latest_path) + + pointer = APIIndexCachePointer.from_json(File.read(latest_path)) + stale_path = File.join(directory_path, "#{pointer.key}.json") + stale_layer = load_matching_layer(stale_path, layer_kind, layer_name, pointer.key) + return CachedAPIIndexLayer.new(nil, "unavailable") unless stale_layer + + CachedAPIIndexLayer.new(stale_layer, "stale") + rescue ex : JSON::ParseException | IO::Error + CachedAPIIndexLayer.new(nil, "unavailable", ex.message) + end + + private def load_matching_layer( + layer_path : String, + expected_kind : String, + expected_name : String, + expected_key : String, + ) : APIIndexLayer? + return nil unless File.file?(layer_path) + + layer = APIIndexLayer.from_json(File.read(layer_path)) + return nil unless layer.layer_kind == expected_kind + return nil unless layer.layer_name == expected_name + return nil unless layer.layer_key == expected_key + + layer + rescue ex : JSON::ParseException | IO::Error + nil + end + + private def layer_directory(layer_kind : String, layer_name : String) : String + case layer_kind + when "stdlib" + File.join(@cache_root, "stdlib") + when "library" + File.join(@cache_root, "libraries", safe_path_component(layer_name)) + else + project_key = Digest::SHA256.hexdigest(File.expand_path(@project_root_path)) + File.join(@cache_root, "projects", project_key, safe_path_component(layer_kind), safe_path_component(layer_name)) + end + end + + private def safe_path_component(value : String) : String + value.gsub(/[^A-Za-z0-9_-]/, "_") + end + + private def write_atomically(path : String, contents : String) : Nil + temp_path = "#{path}.tmp-#{Random::Secure.hex(12)}" + begin + File.write(temp_path, contents) + File.rename(temp_path, path) + rescue ex : IO::Error + if File.file?(temp_path) + File.delete(temp_path) + end + raise ex + end + end + end +end diff --git a/src/amber_lsp/lookup/index_models.cr b/src/amber_lsp/lookup/index_models.cr new file mode 100644 index 0000000..ede044f --- /dev/null +++ b/src/amber_lsp/lookup/index_models.cr @@ -0,0 +1,187 @@ +require "json" + +module AmberLSP::Lookup + # :nodoc: + struct CrystalDocsLocation + include JSON::Serializable + + getter filename : String = "" + @[JSON::Field(key: "line_number")] + getter line_number : Int32 = 0 + getter url : String? = nil + end + + # :nodoc: + struct CrystalDocsTypeReference + include JSON::Serializable + + getter kind : String = "" + getter name : String = "" + @[JSON::Field(key: "full_name")] + getter full_name : String = "" + end + + # :nodoc: + struct CrystalDocsMethodDefinition + include JSON::Serializable + + getter name : String = "" + @[JSON::Field(key: "return_type")] + getter return_type : String? = nil + end + + # :nodoc: + struct CrystalDocsMethod + include JSON::Serializable + + getter name : String = "" + getter? abstract : Bool = false + @[JSON::Field(key: "args_string")] + getter args_string : String? = nil + getter doc : String? = nil + getter location : CrystalDocsLocation? = nil + @[JSON::Field(key: "def")] + getter definition : CrystalDocsMethodDefinition? = nil + end + + # :nodoc: + struct CrystalDocsType + include JSON::Serializable + + getter kind : String = "" + getter name : String = "" + @[JSON::Field(key: "full_name")] + getter full_name : String = "" + getter? abstract : Bool = false + getter locations : Array(CrystalDocsLocation) = [] of CrystalDocsLocation + getter ancestors : Array(CrystalDocsTypeReference) = [] of CrystalDocsTypeReference + @[JSON::Field(key: "included_modules")] + getter included_modules : Array(CrystalDocsTypeReference) = [] of CrystalDocsTypeReference + @[JSON::Field(key: "extended_modules")] + getter extended_modules : Array(CrystalDocsTypeReference) = [] of CrystalDocsTypeReference + @[JSON::Field(key: "instance_methods")] + getter instance_methods : Array(CrystalDocsMethod) = [] of CrystalDocsMethod + @[JSON::Field(key: "class_methods")] + getter class_methods : Array(CrystalDocsMethod) = [] of CrystalDocsMethod + getter macros : Array(CrystalDocsMethod) = [] of CrystalDocsMethod + getter types : Array(CrystalDocsType) = [] of CrystalDocsType + + def initialize( + @kind : String = "", + @name : String = "", + @full_name : String = "", + @abstract : Bool = false, + @locations : Array(CrystalDocsLocation) = [] of CrystalDocsLocation, + @ancestors : Array(CrystalDocsTypeReference) = [] of CrystalDocsTypeReference, + @included_modules : Array(CrystalDocsTypeReference) = [] of CrystalDocsTypeReference, + @extended_modules : Array(CrystalDocsTypeReference) = [] of CrystalDocsTypeReference, + @instance_methods : Array(CrystalDocsMethod) = [] of CrystalDocsMethod, + @class_methods : Array(CrystalDocsMethod) = [] of CrystalDocsMethod, + @macros : Array(CrystalDocsMethod) = [] of CrystalDocsMethod, + @types : Array(CrystalDocsType) = [] of CrystalDocsType, + ) + end + end + + # :nodoc: + struct CrystalDocsExport + include JSON::Serializable + + getter repository_name : String + getter program : CrystalDocsType + + def initialize( + @repository_name : String = "", + @program : CrystalDocsType = CrystalDocsType.new, + ) + end + end + + # :nodoc: + struct APIIndexMethod + include JSON::Serializable + + getter owner : String + getter name : String + getter method_kind : String + getter args_string : String + getter return_type : String + getter doc_line : String? = nil + getter source_path : String + getter source_line : Int32 + getter source_layer : String + getter? abstract : Bool + getter? macro : Bool + + def initialize( + @owner : String, + @name : String, + @method_kind : String, + @args_string : String, + @return_type : String, + @doc_line : String?, + @source_path : String, + @source_line : Int32, + @source_layer : String, + @abstract : Bool, + @macro : Bool, + ) + end + end + + # :nodoc: + struct APIIndexType + include JSON::Serializable + + getter name : String + getter kind : String + getter? abstract : Bool + getter list_of_ancestor_names : Array(String) + getter list_of_included_module_names : Array(String) + getter list_of_extended_module_names : Array(String) + getter location_path : String + getter location_line : Int32 + getter source_layer : String + getter list_of_instance_methods : Array(APIIndexMethod) + getter list_of_class_methods : Array(APIIndexMethod) + getter list_of_macros : Array(APIIndexMethod) + + def initialize( + @name : String, + @kind : String, + @abstract : Bool, + @list_of_ancestor_names : Array(String), + @list_of_included_module_names : Array(String), + @list_of_extended_module_names : Array(String), + @location_path : String, + @location_line : Int32, + @source_layer : String, + @list_of_instance_methods : Array(APIIndexMethod), + @list_of_class_methods : Array(APIIndexMethod), + @list_of_macros : Array(APIIndexMethod), + ) + end + end + + # :nodoc: + struct APIIndexLayer + include JSON::Serializable + + getter layer_kind : String + getter layer_name : String + getter layer_key : String + getter root_path : String + getter docs_flags : Array(String) + getter list_of_types : Array(APIIndexType) + + def initialize( + @layer_kind : String, + @layer_name : String, + @layer_key : String, + @root_path : String, + @docs_flags : Array(String), + @list_of_types : Array(APIIndexType), + ) + end + end +end diff --git a/src/amber_lsp/lookup/normalize_crystal_docs.cr b/src/amber_lsp/lookup/normalize_crystal_docs.cr new file mode 100644 index 0000000..2913ebb --- /dev/null +++ b/src/amber_lsp/lookup/normalize_crystal_docs.cr @@ -0,0 +1,111 @@ +require "./index_models" + +module AmberLSP::Lookup + class NormalizeCrystalDocs + def initialize( + @docs_json : String, + @root_path : String, + @layer_kind : String, + @layer_name : String, + @layer_key : String, + @docs_flags : Array(String), + ) + end + + def perform : APIIndexLayer + export = CrystalDocsExport.from_json(@docs_json) + list_of_types = [] of APIIndexType + append_types_from(export.program, list_of_types) + + APIIndexLayer.new( + @layer_kind, + @layer_name, + @layer_key, + @root_path, + @docs_flags, + list_of_types, + ) + end + + private def append_types_from(parent : CrystalDocsType, list_of_types : Array(APIIndexType)) : Nil + parent.types.each do |docs_type| + next if docs_type.locations.empty? + + list_of_types << normalize_type(docs_type) + append_types_from(docs_type, list_of_types) + end + end + + private def normalize_type(docs_type : CrystalDocsType) : APIIndexType + type_location = docs_type.locations.first? + location_path = type_location ? source_path(type_location.filename) : "" + location_line = type_location ? type_location.line_number : 0 + + APIIndexType.new( + docs_type.full_name, + docs_type.kind, + docs_type.abstract?, + docs_type.ancestors.map(&.full_name), + docs_type.included_modules.map(&.full_name), + docs_type.extended_modules.map(&.full_name), + location_path, + location_line, + @layer_name, + normalize_methods(docs_type, docs_type.instance_methods, "instance", false), + normalize_methods(docs_type, docs_type.class_methods, "class", false), + normalize_methods(docs_type, docs_type.macros, "macro", true), + ) + end + + private def normalize_methods( + docs_type : CrystalDocsType, + docs_methods : Array(CrystalDocsMethod), + method_kind : String, + is_macro : Bool, + ) : Array(APIIndexMethod) + docs_methods.compact_map do |docs_method| + location = docs_method.location + next unless location + + doc_line = first_doc_line(docs_method.doc) + definition = docs_method.definition + return_type = definition ? definition.return_type : nil + + APIIndexMethod.new( + docs_type.full_name, + docs_method.name, + method_kind, + method_arguments(docs_method.args_string), + return_type || "unknown", + doc_line, + source_path(location.filename), + location.line_number, + @layer_name, + docs_method.abstract?, + is_macro, + ) + end + end + + private def method_arguments(args_string : String?) : String + return "()" unless args_string + + args_string.sub(/\)\s*:\s*.+\z/, ")") + end + + private def first_doc_line(doc : String?) : String? + return nil unless doc + + first_line = doc.split('\n', 2).first?.try(&.strip) + return nil if first_line.nil? || first_line.empty? + + first_line + end + + private def source_path(filename : String) : String + return filename if Path[filename].absolute? + + File.expand_path(filename, @root_path) + end + end +end diff --git a/src/amber_lsp/lookup/source_models.cr b/src/amber_lsp/lookup/source_models.cr new file mode 100644 index 0000000..18d6a42 --- /dev/null +++ b/src/amber_lsp/lookup/source_models.cr @@ -0,0 +1,47 @@ +require "yaml" + +module AmberLSP::Lookup + # :nodoc: + struct ProjectShardTarget + include YAML::Serializable + + getter main : String = "" + end + + # :nodoc: + struct ProjectShardDependency + include YAML::Serializable + end + + # :nodoc: + struct ProjectShardManifest + include YAML::Serializable + + getter name : String = "" + getter version : String = "0.0.0" + getter dependencies : Hash(String, ProjectShardDependency) = {} of String => ProjectShardDependency + getter targets : Hash(String, ProjectShardTarget) = {} of String => ProjectShardTarget + + def initialize( + @name : String = "", + @version : String = "0.0.0", + @dependencies : Hash(String, ProjectShardDependency) = {} of String => ProjectShardDependency, + @targets : Hash(String, ProjectShardTarget) = {} of String => ProjectShardTarget, + ) + end + end + + # :nodoc: + struct LockedShard + include YAML::Serializable + + getter version : String = "" + end + + # :nodoc: + struct ShardLockfile + include YAML::Serializable + + getter shards : Hash(String, LockedShard) = {} of String => LockedShard + end +end From db77bb255e30df23d4b0c05fa2b61e1335455f81 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:01:34 -0400 Subject: [PATCH 20/69] feat(lsp): resolve Crystal API lookup queries --- .../lookup/resolve_api_query_spec.cr | 82 +++++++ .../api_lookup/crystal_docs_small.json | 72 +++++- src/amber_lsp.cr | 1 + src/amber_lsp/lookup/resolve_api_query.cr | 217 ++++++++++++++++++ 4 files changed, 371 insertions(+), 1 deletion(-) create mode 100644 spec/amber_lsp/lookup/resolve_api_query_spec.cr create mode 100644 src/amber_lsp/lookup/resolve_api_query.cr diff --git a/spec/amber_lsp/lookup/resolve_api_query_spec.cr b/spec/amber_lsp/lookup/resolve_api_query_spec.cr new file mode 100644 index 0000000..2f9646c --- /dev/null +++ b/spec/amber_lsp/lookup/resolve_api_query_spec.cr @@ -0,0 +1,82 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/normalize_crystal_docs" +require "../../../src/amber_lsp/lookup/resolve_api_query" + +describe "AmberLSP::Lookup::ResolveAPIQuery#perform" do + it "resolves a type's own class method before its ancestor method" do + result = resolve_fixture_query("FixtureAPI::User.count") + + result.resolution_kind.should eq("class_method") + result.list_of_methods.size.should eq(1) + result.list_of_methods.first.owner.should eq("FixtureAPI::User") + result.list_of_methods.first.return_type.should eq("Int32") + end + + it "resolves class methods and instance methods through ancestors" do + class_result = resolve_fixture_query("FixtureAPI::Team.count") + instance_result = resolve_fixture_query("FixtureAPI::Team#created_at") + + class_result.list_of_methods.first.owner.should eq("Grant::Base") + class_result.list_of_methods.first.return_type.should eq("Int64") + instance_result.list_of_methods.first.owner.should eq("Grant::Base") + instance_result.list_of_methods.first.name.should eq("created_at") + end + + it "resolves methods extended directly on a type and on its ancestor" do + direct_result = resolve_fixture_query("FixtureAPI::User.where") + ancestor_result = resolve_fixture_query("FixtureAPI::Team.where") + + direct_result.resolution_kind.should eq("class_method") + direct_result.list_of_methods.size.should eq(2) + direct_result.list_of_methods.all? { |method| method.owner == "FixtureAPI::QueryMethods" }.should be_true + ancestor_result.list_of_methods.size.should eq(2) + ancestor_result.list_of_methods.all? { |method| method.owner == "FixtureAPI::QueryMethods" }.should be_true + end + + it "matches generic extension owners by their base name" do + result = resolve_fixture_query("FixtureAPI::Team.build") + + result.list_of_methods.size.should eq(1) + result.list_of_methods.first.owner.should eq("Grant::Query::Builder(Model)") + end + + it "returns a type summary for a type-only query, including a generic base-name match" do + result = resolve_fixture_query("FixtureAPI::Team") + generic_result = resolve_fixture_query("Grant::Query::Builder") + + result.resolution_kind.should eq("type") + result.type_summary.should_not be_nil + result.type_summary.not_nil!.name.should eq("FixtureAPI::Team") + generic_result.resolution_kind.should eq("type") + generic_result.type_summary.not_nil!.name.should eq("Grant::Query::Builder(Model)") + end + + it "returns bare method candidates from all owners" do + result = resolve_fixture_query("where") + + result.resolution_kind.should eq("bare_method") + result.list_of_methods.size.should eq(2) + result.list_of_methods.all? { |method| method.owner == "FixtureAPI::QueryMethods" }.should be_true + end + + it "returns unknown when no type or method matches" do + result = resolve_fixture_query("Post.find_by_sql") + + result.resolution_kind.should eq("unknown") + result.list_of_methods.should be_empty + result.type_summary.should be_nil + end +end + +private def resolve_fixture_query(query : String) : AmberLSP::Lookup::APIResolution + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_small.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + "/project", + "project", + "fixture_api", + "fixture-key", + [] of String, + ).perform + AmberLSP::Lookup::ResolveAPIQuery.new(query, [layer]).perform +end diff --git a/spec/fixtures/api_lookup/crystal_docs_small.json b/spec/fixtures/api_lookup/crystal_docs_small.json index 0dacd7e..7da2b52 100644 --- a/spec/fixtures/api_lookup/crystal_docs_small.json +++ b/spec/fixtures/api_lookup/crystal_docs_small.json @@ -25,7 +25,16 @@ "def": {"name": "id", "return_type": "Int64 | ::Nil"} } ], - "class_methods": [], + "class_methods": [ + { + "name": "count", + "abstract": false, + "args_string": "() : Int32", + "doc": "Counts users.", + "location": {"filename": "src/models/user.cr", "line_number": 20, "url": null}, + "def": {"name": "count", "return_type": "Int32"} + } + ], "macros": [ { "name": "has_token", @@ -37,6 +46,50 @@ } ] }, + { + "kind": "class", + "name": "Team", + "full_name": "FixtureAPI::Team", + "abstract": false, + "locations": [{"filename": "src/models/team.cr", "line_number": 3}], + "ancestors": [{"kind": "class", "name": "Base", "full_name": "Grant::Base"}], + "instance_methods": [], + "class_methods": [], + "macros": [] + }, + { + "kind": "class", + "name": "Base", + "full_name": "Grant::Base", + "abstract": true, + "locations": [{"filename": "lib/grant/src/grant/base.cr", "line_number": 10}], + "ancestors": [], + "extended_modules": [ + {"kind": "module", "name": "QueryMethods", "full_name": "FixtureAPI::QueryMethods"}, + {"kind": "module", "name": "Builder", "full_name": "Grant::Query::Builder"} + ], + "instance_methods": [ + { + "name": "created_at", + "abstract": false, + "args_string": "() : Time", + "doc": "Returns the creation time.", + "location": {"filename": "lib/grant/src/grant/base.cr", "line_number": 44, "url": null}, + "def": {"name": "created_at", "return_type": "Time"} + } + ], + "class_methods": [ + { + "name": "count", + "abstract": false, + "args_string": "() : Int64", + "doc": "Counts all rows.", + "location": {"filename": "lib/grant/src/grant/base.cr", "line_number": 65, "url": null}, + "def": {"name": "count", "return_type": "Int64"} + } + ], + "macros": [] + }, { "kind": "module", "name": "QueryMethods", @@ -61,6 +114,23 @@ "def": {"name": "where", "return_type": "Query"} } ] + }, + { + "kind": "module", + "name": "Builder", + "full_name": "Grant::Query::Builder(Model)", + "abstract": false, + "locations": [{"filename": "lib/grant/src/grant/query/builder.cr", "line_number": 1}], + "instance_methods": [ + { + "name": "build", + "abstract": false, + "args_string": "(condition : String) : Query", + "doc": "Builds a query.", + "location": {"filename": "lib/grant/src/grant/query/builder.cr", "line_number": 27, "url": null}, + "def": {"name": "build", "return_type": "Query"} + } + ] } ] } diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 5d18a5f..9b8b9d6 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -42,6 +42,7 @@ require "./amber_lsp/controller" require "./amber_lsp/server" require "./amber_lsp/check_file_for_diagnostics" require "./amber_lsp/lookup/build_layered_api_index" +require "./amber_lsp/lookup/resolve_api_query" if ARGV.includes?("--version") STDOUT.puts AmberLSP.version_line diff --git a/src/amber_lsp/lookup/resolve_api_query.cr b/src/amber_lsp/lookup/resolve_api_query.cr new file mode 100644 index 0000000..e374273 --- /dev/null +++ b/src/amber_lsp/lookup/resolve_api_query.cr @@ -0,0 +1,217 @@ +require "set" + +require "./index_models" + +module AmberLSP::Lookup + struct APIResolution + getter query : String + getter resolution_kind : String + getter type_summary : APIIndexType? + getter list_of_methods : Array(APIIndexMethod) + + def initialize( + @query : String, + @resolution_kind : String, + @type_summary : APIIndexType? = nil, + @list_of_methods : Array(APIIndexMethod) = [] of APIIndexMethod, + ) + end + end + + class ResolveAPIQuery + def initialize(@query : String, @list_of_layers : Array(APIIndexLayer)) + end + + def perform : APIResolution + query = @query.strip + return unknown_result(query) if query.empty? + + if match = query.match(/\A(.+)#([^.#]+)\z/) + return resolve_instance_method(query, match[1], match[2]) + end + + if match = query.match(/\A(.+)\.([^.#]+)\z/) + return resolve_class_method(query, match[1], match[2]) + end + + if query[0].uppercase? + return resolve_type(query) + end + + resolve_bare_method(query) + end + + private def resolve_type(query : String) : APIResolution + matching_types = types_named(query) + type_summary = matching_types.first? + return unknown_result(query) unless type_summary + + APIResolution.new(query, "type", type_summary) + end + + private def resolve_instance_method(query : String, type_name : String, method_name : String) : APIResolution + matching_types = types_named(type_name) + return unknown_result(query) if matching_types.empty? + + methods = methods_named(matching_types, method_name, "instance") + return found_result(query, "instance_method", methods) unless methods.empty? + + ancestor_types_for(matching_types).each do |ancestor_type| + methods = methods_named([ancestor_type], method_name, "instance") + return found_result(query, "instance_method", methods) unless methods.empty? + end + + unknown_result(query) + end + + private def resolve_class_method(query : String, type_name : String, method_name : String) : APIResolution + matching_types = types_named(type_name) + return unknown_result(query) if matching_types.empty? + + methods = methods_named(matching_types, method_name, "class") + return found_result(query, "class_method", methods) unless methods.empty? + + ancestor_types = ancestor_types_for(matching_types) + ancestor_types.each do |ancestor_type| + methods = methods_named([ancestor_type], method_name, "class") + return found_result(query, "class_method", methods) unless methods.empty? + end + + methods = extended_module_methods(matching_types + ancestor_types, method_name) + return found_result(query, "class_method", methods) unless methods.empty? + + macro_methods = methods_named(matching_types, method_name, "macro") + return found_result(query, "class_method", macro_methods) unless macro_methods.empty? + + unknown_result(query) + end + + private def resolve_bare_method(query : String) : APIResolution + methods = [] of APIIndexMethod + all_types.each do |type| + methods.concat(methods_named([type], query, "instance")) + methods.concat(methods_named([type], query, "class")) + methods.concat(methods_named([type], query, "macro")) + end + + APIResolution.new(query, "bare_method", nil, unique_methods(methods)) + end + + private def extended_module_methods(owner_types : Array(APIIndexType), method_name : String) : Array(APIIndexMethod) + methods = [] of APIIndexMethod + visited_module_names = Set(String).new + + owner_types.each do |owner_type| + owner_type.list_of_extended_module_names.each do |module_name| + next if visited_module_names.includes?(module_name) + + visited_module_names.add(module_name) + types_named(module_name).each do |module_type| + methods.concat(methods_named([module_type], method_name, "instance")) + end + end + end + + unique_methods(methods) + end + + private def ancestor_types_for(types : Array(APIIndexType)) : Array(APIIndexType) + list_of_ancestors = [] of APIIndexType + types.each do |type| + type.list_of_ancestor_names.each do |ancestor_name| + list_of_ancestors.concat(types_named(ancestor_name)) + end + end + unique_types(list_of_ancestors) + end + + private def methods_named(types : Array(APIIndexType), method_name : String, method_kind : String) : Array(APIIndexMethod) + methods = [] of APIIndexMethod + types.each do |type| + case method_kind + when "instance" + methods.concat(type.list_of_instance_methods.select { |method| method.name == method_name }) + when "class" + methods.concat(type.list_of_class_methods.select { |method| method.name == method_name }) + when "macro" + methods.concat(type.list_of_macros.select { |method| method.name == method_name }) + end + end + unique_methods(methods) + end + + private def types_named(name : String) : Array(APIIndexType) + normalized_name = normalize_type_name(name) + matching_types = all_types.select do |type| + normalize_type_name(type.name) == normalized_name + end + return matching_types unless matching_types.empty? + + base_name = base_type_name(normalized_name) + all_types.select { |type| base_type_name(normalize_type_name(type.name)) == base_name } + end + + private def all_types : Array(APIIndexType) + @list_of_layers.flat_map(&.list_of_types) + end + + private def unique_types(types : Array(APIIndexType)) : Array(APIIndexType) + seen = Set(String).new + types.select do |type| + type_key = "#{type.name}\0#{type.source_layer}\0#{type.location_path}\0#{type.location_line}" + next false if seen.includes?(type_key) + + seen.add(type_key) + true + end + end + + private def unique_methods(methods : Array(APIIndexMethod)) : Array(APIIndexMethod) + seen = Set(String).new + methods.select do |method| + method_key = [ + method.owner, + method.name, + method.method_kind, + method.args_string, + method.source_path, + method.source_line.to_s, + ].join("\0") + next false if seen.includes?(method_key) + + seen.add(method_key) + true + end + end + + private def normalize_type_name(name : String) : String + name.sub(/\A::/, "") + end + + private def base_type_name(name : String) : String + output = String::Builder.new + generic_depth = 0 + + name.each_char do |character| + case character + when '(' + generic_depth += 1 + when ')' + generic_depth -= 1 if generic_depth > 0 + else + output << character unless generic_depth > 0 + end + end + + output.to_s + end + + private def found_result(query : String, resolution_kind : String, methods : Array(APIIndexMethod)) : APIResolution + APIResolution.new(query, resolution_kind, nil, methods) + end + + private def unknown_result(query : String) : APIResolution + APIResolution.new(query, "unknown") + end + end +end From a9b4e6b44202e30b70551ebfdd68ade070b5c40d Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:05:26 -0400 Subject: [PATCH 21/69] feat(lsp): report lookup status and freshness --- .../amber_lsp/lookup/answer_api_query_spec.cr | 91 +++++++++++++++ src/amber_lsp.cr | 1 + src/amber_lsp/lookup/answer_api_query.cr | 106 ++++++++++++++++++ .../lookup/build_layered_api_index.cr | 24 +++- src/amber_lsp/lookup/index_cache.cr | 27 ++++- 5 files changed, 241 insertions(+), 8 deletions(-) create mode 100644 spec/amber_lsp/lookup/answer_api_query_spec.cr create mode 100644 src/amber_lsp/lookup/answer_api_query.cr diff --git a/spec/amber_lsp/lookup/answer_api_query_spec.cr b/spec/amber_lsp/lookup/answer_api_query_spec.cr new file mode 100644 index 0000000..5b27688 --- /dev/null +++ b/spec/amber_lsp/lookup/answer_api_query_spec.cr @@ -0,0 +1,91 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/answer_api_query" +require "../../../src/amber_lsp/lookup/normalize_crystal_docs" +require "../../../src/amber_lsp/lookup/resolve_api_query" + +describe "AmberLSP::Lookup::AnswerAPIQuery#perform" do + it "returns found with fresh source provenance for an indexed method" do + answer = build_fixture_answer("FixtureAPI::User#id", "fresh") + + answer.status.should eq("found") + answer.freshness.should eq("fresh") + answer.list_of_entries.first.source_layer.should eq("fixture_api") + answer.list_of_layers.first.layer_name.should eq("fixture_api") + end + + it "returns candidates for a bare method query" do + answer = build_fixture_answer("where", "fresh") + + answer.status.should eq("candidates") + answer.list_of_entries.size.should eq(2) + end + + it "returns unknown with fresh freshness without claiming absence" do + answer = build_fixture_answer("Post.find_by_sql", "fresh") + json = answer.to_json + + answer.status.should eq("unknown") + answer.freshness.should eq("fresh") + json.should_not contain("absent") + end + + it "preserves stale freshness when it answers from a previous layer" do + answer = build_fixture_answer("FixtureAPI::User#id", "stale") + + answer.status.should eq("found") + answer.freshness.should eq("stale") + answer.list_of_entries.first.source_layer.should eq("fixture_api") + end + + it "returns unavailable freshness when any required layer is unavailable" do + answer = build_fixture_answer("FixtureAPI::User#id", "fresh", missing_library: true) + + answer.status.should eq("found") + answer.freshness.should eq("unavailable") + answer.list_of_layers.map(&.freshness).should contain("unavailable") + end + + it "keeps unknown unknown when a stale index has no matching method" do + answer = build_fixture_answer("Post.find_by_sql", "stale") + + answer.status.should eq("unknown") + answer.freshness.should eq("stale") + end + + it "marks an unknown answer unavailable if a required library layer is missing" do + answer = build_fixture_answer("Post.find_by_sql", "fresh", missing_library: true) + + answer.status.should eq("unknown") + answer.freshness.should eq("unavailable") + end +end + +private def build_fixture_answer( + query : String, + freshness : String, + missing_library : Bool = false, +) : AmberLSP::Lookup::LookupAnswer + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_small.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + "/project", + "project", + "fixture_api", + "fixture-key", + [] of String, + ).perform + list_of_layers = [AmberLSP::Lookup::CachedAPIIndexLayer.new(layer, freshness)] + if missing_library + list_of_layers << AmberLSP::Lookup::CachedAPIIndexLayer.new( + nil, + "unavailable", + "locked shard is missing", + "library", + "grant", + "", + ) + end + + resolution = AmberLSP::Lookup::ResolveAPIQuery.new(query, [layer]).perform + AmberLSP::Lookup::AnswerAPIQuery.new(query, resolution, list_of_layers).perform +end diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 9b8b9d6..707c2ef 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -43,6 +43,7 @@ require "./amber_lsp/server" require "./amber_lsp/check_file_for_diagnostics" require "./amber_lsp/lookup/build_layered_api_index" require "./amber_lsp/lookup/resolve_api_query" +require "./amber_lsp/lookup/answer_api_query" if ARGV.includes?("--version") STDOUT.puts AmberLSP.version_line diff --git a/src/amber_lsp/lookup/answer_api_query.cr b/src/amber_lsp/lookup/answer_api_query.cr new file mode 100644 index 0000000..b9046c9 --- /dev/null +++ b/src/amber_lsp/lookup/answer_api_query.cr @@ -0,0 +1,106 @@ +require "json" + +require "./index_cache" +require "./index_models" +require "./resolve_api_query" + +module AmberLSP::Lookup + struct APIIndexLayerState + include JSON::Serializable + + getter layer_kind : String + getter layer_name : String + getter layer_key : String + getter freshness : String + @[JSON::Field(emit_null: true)] + getter failure_reason : String? + + def initialize( + @layer_kind : String, + @layer_name : String, + @layer_key : String, + @freshness : String, + @failure_reason : String?, + ) + end + end + + struct LookupAnswer + include JSON::Serializable + + getter query : String + getter status : String + getter freshness : String + @[JSON::Field(emit_null: true)] + getter type_summary : APIIndexType? + @[JSON::Field(key: "entries")] + getter list_of_entries : Array(APIIndexMethod) + @[JSON::Field(key: "layers")] + getter list_of_layers : Array(APIIndexLayerState) + + def initialize( + @query : String, + @status : String, + @freshness : String, + @type_summary : APIIndexType?, + @list_of_entries : Array(APIIndexMethod), + @list_of_layers : Array(APIIndexLayerState), + ) + end + end + + class AnswerAPIQuery + def initialize( + @query : String, + @resolution : APIResolution, + @list_of_layers : Array(CachedAPIIndexLayer), + ) + end + + def perform : LookupAnswer + answer_status = status + answer_freshness = freshness + list_of_layer_states = @list_of_layers.map do |cached_layer| + APIIndexLayerState.new( + cached_layer.layer_kind, + cached_layer.layer_name, + cached_layer.layer_key, + cached_layer.freshness, + cached_layer.failure_reason, + ) + end + + LookupAnswer.new( + @query, + answer_status, + answer_freshness, + @resolution.type_summary, + @resolution.list_of_methods, + list_of_layer_states, + ) + end + + private def status : String + if @resolution.resolution_kind == "type" && @resolution.type_summary + return "found" + end + + if @resolution.resolution_kind == "bare_method" + return @resolution.list_of_methods.empty? ? "unknown" : "candidates" + end + + @resolution.list_of_methods.empty? ? "unknown" : "found" + end + + private def freshness : String + return "unavailable" if @list_of_layers.empty? + + return "unavailable" if @list_of_layers.any? { |layer| layer.freshness == "unavailable" } + return "stale" if @list_of_layers.any? { |layer| layer.freshness == "stale" } + + return "fresh" if @list_of_layers.all? { |layer| layer.freshness == "fresh" } + + "unavailable" + end + end +end diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr index 4c5e80a..30ac288 100644 --- a/src/amber_lsp/lookup/build_layered_api_index.cr +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -93,19 +93,35 @@ module AmberLSP::Lookup nil, "unavailable", "#{library_name} is declared in shard.yml but is missing from shard.lock", + "library", + library_name, ) end locked_shards.to_a.sort_by(&.first).each do |library_name, locked_shard| library_root = File.join(@root_path, "lib", library_name) unless File.directory?(library_root) - list_of_layers << CachedAPIIndexLayer.new(nil, "unavailable", "Locked shard #{library_name} is not installed under lib/") + list_of_layers << CachedAPIIndexLayer.new( + nil, + "unavailable", + "Locked shard #{library_name} is not installed under lib/", + "library", + library_name, + locked_shard.version, + ) next end entrypoint = find_library_entrypoint(library_root, library_name) unless entrypoint - list_of_layers << CachedAPIIndexLayer.new(nil, "unavailable", "Locked shard #{library_name} has no Crystal entrypoint under lib/#{library_name}/src") + list_of_layers << CachedAPIIndexLayer.new( + nil, + "unavailable", + "Locked shard #{library_name} has no Crystal entrypoint under lib/#{library_name}/src", + "library", + library_name, + locked_shard.version, + ) next end @@ -226,7 +242,7 @@ module AmberLSP::Lookup rescue ex : APIIndexBuildError | JSON::ParseException return cached_layer if cached_layer.layer - CachedAPIIndexLayer.new(nil, "unavailable", ex.message) + CachedAPIIndexLayer.new(nil, "unavailable", ex.message, layer_kind, layer_name, layer_key) end end @@ -265,7 +281,7 @@ module AmberLSP::Lookup rescue ex : APIIndexBuildError | JSON::ParseException | IO::Error return cached_layer if cached_layer.layer - CachedAPIIndexLayer.new(nil, "unavailable", ex.message) + CachedAPIIndexLayer.new(nil, "unavailable", ex.message, "stdlib", "crystal", layer_key) end end diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr index 69e89a2..b094426 100644 --- a/src/amber_lsp/lookup/index_cache.cr +++ b/src/amber_lsp/lookup/index_cache.cr @@ -76,8 +76,27 @@ module AmberLSP::Lookup getter layer : APIIndexLayer? getter freshness : String getter failure_reason : String? + getter layer_kind : String + getter layer_name : String + getter layer_key : String - def initialize(@layer : APIIndexLayer?, @freshness : String, @failure_reason : String? = nil) + def initialize( + @layer : APIIndexLayer?, + @freshness : String, + @failure_reason : String? = nil, + layer_kind : String? = nil, + layer_name : String? = nil, + layer_key : String? = nil, + ) + if index_layer = @layer + @layer_kind = layer_kind || index_layer.layer_kind + @layer_name = layer_name || index_layer.layer_name + @layer_key = layer_key || index_layer.layer_key + else + @layer_kind = layer_kind || "" + @layer_name = layer_name || "" + @layer_key = layer_key || "" + end end end @@ -122,16 +141,16 @@ module AmberLSP::Lookup end latest_path = File.join(directory_path, "latest.json") - return CachedAPIIndexLayer.new(nil, "unavailable") unless File.file?(latest_path) + return CachedAPIIndexLayer.new(nil, "unavailable", nil, layer_kind, layer_name, expected_key) unless File.file?(latest_path) pointer = APIIndexCachePointer.from_json(File.read(latest_path)) stale_path = File.join(directory_path, "#{pointer.key}.json") stale_layer = load_matching_layer(stale_path, layer_kind, layer_name, pointer.key) - return CachedAPIIndexLayer.new(nil, "unavailable") unless stale_layer + return CachedAPIIndexLayer.new(nil, "unavailable", nil, layer_kind, layer_name, expected_key) unless stale_layer CachedAPIIndexLayer.new(stale_layer, "stale") rescue ex : JSON::ParseException | IO::Error - CachedAPIIndexLayer.new(nil, "unavailable", ex.message) + CachedAPIIndexLayer.new(nil, "unavailable", ex.message, layer_kind, layer_name, expected_key) end private def load_matching_layer( From 105305b2b67915c775f1b43d993cbc45a5afd6be Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:15:26 -0400 Subject: [PATCH 22/69] feat(lsp): verify Crystal API lookups with compiler probes --- .../amber_lsp/lookup/verify_api_query_spec.cr | 121 ++++++++ src/amber_lsp.cr | 1 + src/amber_lsp/lookup/verify_api_query.cr | 266 ++++++++++++++++++ 3 files changed, 388 insertions(+) create mode 100644 spec/amber_lsp/lookup/verify_api_query_spec.cr create mode 100644 src/amber_lsp/lookup/verify_api_query.cr diff --git a/spec/amber_lsp/lookup/verify_api_query_spec.cr b/spec/amber_lsp/lookup/verify_api_query_spec.cr new file mode 100644 index 0000000..d14708d --- /dev/null +++ b/spec/amber_lsp/lookup/verify_api_query_spec.cr @@ -0,0 +1,121 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/verify_api_query" + +describe AmberLSP::Lookup::VerifyAPIQuery do + it "upgrades unknown instance and inherited class methods to present" do + with_probe_project do |project| + instance_result = verify_query(project, "ProbeAPI::Child#inherited_api") + class_result = verify_query(project, "ProbeAPI::Child.inherited_class_api") + extension_result = verify_query(project, "ProbeAPI::Child.extended_class_api") + instance_result.status.should eq("present") + class_result.status.should eq("present") + extension_result.status.should eq("present") + instance_result.elapsed_milliseconds.should be >= 0 + end + end + + it "upgrades unknown methods and missing types to absent" do + with_probe_project do |project| + method_result = verify_query(project, "ProbeAPI::Child#missing_api") + type_result = verify_query(project, "ProbeAPI::MissingType") + + method_result.status.should eq("absent") + method_result.output.should contain("AMBER_LSP_PROBE_ABSENT") + type_result.status.should eq("absent") + end + end + + it "narrows bare method candidates to the first verified owner" do + with_probe_project do |project| + absent_candidate = probe_method("ProbeAPI::FirstCandidate", "where") + present_candidate = probe_method("ProbeAPI::SecondCandidate", "where") + resolution = AmberLSP::Lookup::APIResolution.new( + "where", + "bare_method", + nil, + [absent_candidate, present_candidate], + ) + + result = AmberLSP::Lookup::VerifyAPIQuery.new(project, "where", resolution).perform + + result.status.should eq("present") + result.verified_entry.should_not be_nil + result.verified_entry.not_nil!.owner.should eq("ProbeAPI::SecondCandidate") + end + end + + it "returns unavailable when the query cannot be represented safely" do + with_probe_project do |project| + result = verify_query(project, "ProbeAPI::Child#safe_name; puts 1") + + result.status.should eq("unavailable") + end + end +end + +private def verify_query(project : String, query : String) : AmberLSP::Lookup::APIProbeResult + resolution = AmberLSP::Lookup::APIResolution.new(query, "unknown") + AmberLSP::Lookup::VerifyAPIQuery.new(project, query, resolution).perform +end + +private def probe_method(owner : String, name : String) : AmberLSP::Lookup::APIIndexMethod + AmberLSP::Lookup::APIIndexMethod.new( + owner, + name, + "instance", + "", + "", + nil, + "src/probe_api.cr", + 1, + "fixture", + false, + false, + ) +end + +private def with_probe_project(&) + with_tempdir do |project| + Dir.mkdir_p(File.join(project, "src")) + File.write(File.join(project, "shard.yml"), <<-YAML) + name: amber_lsp_probe_fixture + version: 0.1.0 + targets: + amber_lsp_probe_fixture: + main: src/amber_lsp_probe_fixture.cr + YAML + File.write(File.join(project, "src", "amber_lsp_probe_fixture.cr"), <<-CRYSTAL) + module ProbeAPI + module ClassExtensions + def extended_class_api + end + end + + module InstanceExtensions + def where + end + end + + class Parent + def inherited_api + end + + def self.inherited_class_api + end + end + + class Child < Parent + extend ClassExtensions + end + + class FirstCandidate + end + + class SecondCandidate + include InstanceExtensions + end + end + CRYSTAL + yield project + end +end diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 707c2ef..652ab37 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -44,6 +44,7 @@ require "./amber_lsp/check_file_for_diagnostics" require "./amber_lsp/lookup/build_layered_api_index" require "./amber_lsp/lookup/resolve_api_query" require "./amber_lsp/lookup/answer_api_query" +require "./amber_lsp/lookup/verify_api_query" if ARGV.includes?("--version") STDOUT.puts AmberLSP.version_line diff --git a/src/amber_lsp/lookup/verify_api_query.cr b/src/amber_lsp/lookup/verify_api_query.cr new file mode 100644 index 0000000..372db06 --- /dev/null +++ b/src/amber_lsp/lookup/verify_api_query.cr @@ -0,0 +1,266 @@ +require "file_utils" +require "process" +require "path" +require "random/secure" +require "set" +require "yaml" + +require "./index_models" +require "./index_cache" +require "./resolve_api_query" +require "./source_models" + +module AmberLSP::Lookup + struct APIProbeResult + getter status : String + getter elapsed_milliseconds : Int64 + getter output : String + getter verified_entry : APIIndexMethod? + + def initialize( + @status : String, + @elapsed_milliseconds : Int64, + @output : String, + @verified_entry : APIIndexMethod? = nil, + ) + end + end + + class VerifyAPIQuery + def initialize( + @project_root_path : String, + @query : String, + @resolution : APIResolution, + @compiler_command : String = "crystal-alpha", + ) + @project_root_path = File.expand_path(@project_root_path) + end + + def perform : APIProbeResult + started_at = Time.instant + compiler_path = Process.find_executable(@compiler_command) + return unavailable(started_at, "crystal-alpha is not available on PATH") unless compiler_path + + target = verification_target + if target + return run_probe(started_at, compiler_path, target) + end + + return verify_candidates(started_at, compiler_path) if @resolution.resolution_kind == "bare_method" + + unavailable(started_at, "The query cannot be represented safely in a Crystal probe") + rescue ex : Exception + APIProbeResult.new("unavailable", elapsed_milliseconds(started_at || Time.instant), ex.message || "Could not run the Crystal API probe") + end + + private def verification_target : VerificationTarget? + if match = @query.strip.match(/\A(.+)#([^.#]+)\z/) + return VerificationTarget.new(match[1], match[2], "instance", nil) if safe_target?(match[1], match[2]) + return nil + end + + if match = @query.strip.match(/\A(.+)\.([^.#]+)\z/) + return VerificationTarget.new(match[1], match[2], "class", nil) if safe_target?(match[1], match[2]) + return nil + end + + if @query.strip[0]?.try(&.uppercase?) + type_name = @query.strip + return VerificationTarget.new(type_name, nil, "type", nil) if safe_type_name?(type_name) + return nil + end + + nil + end + + private def candidate_targets : Array(VerificationTarget) + targets = [] of VerificationTarget + seen_targets = Set(String).new + @resolution.list_of_methods.each do |entry| + next unless entry.name == @query.strip + next if entry.method_kind == "macro" + next unless safe_target?(entry.owner, entry.name) + + target_key = [entry.owner, entry.name, entry.method_kind].join("\0") + next if seen_targets.includes?(target_key) + + seen_targets.add(target_key) + targets << VerificationTarget.new(entry.owner, entry.name, entry.method_kind, entry) + end + targets + end + + private def verify_candidates(started_at : Time::Instant, compiler_path : String) : APIProbeResult + targets = candidate_targets + return unavailable(started_at, "The index returned no safe candidates for this query") if targets.empty? + + targets.each do |target| + result = run_probe(started_at, compiler_path, target) + return result if result.status == "present" || result.status == "unavailable" + end + + unavailable(started_at, "No indexed candidate was verified; the compiler cannot establish global absence for a bare method") + end + + private def run_probe( + started_at : Time::Instant, + compiler_path : String, + target : VerificationTarget, + ) : APIProbeResult + probe_root : String? = nil + entrypoint = project_entrypoint + probe_root = File.join(@project_root_path, ".amber-lsp-probe-#{Random::Secure.hex(10)}") + cache_path = File.join(probe_root, "cache") + probe_path = File.join(probe_root, "probe.cr") + Dir.mkdir_p(cache_path) + relative_entrypoint = Path[entrypoint].relative_to(@project_root_path).to_s + require_path = File.join("..", relative_entrypoint).sub(/\.cr\z/, "") + File.write(probe_path, probe_source(require_path, target)) + + output = IO::Memory.new + error_output = IO::Memory.new + status = Process.run( + compiler_path, + ["build", "--no-codegen", probe_path], + chdir: @project_root_path, + env: {"CRYSTAL_CACHE_DIR" => cache_path}, + output: output, + error: error_output, + ) + compiler_output = [output.to_s, error_output.to_s].reject(&.empty?).join("\n").strip + + if status.success? + APIProbeResult.new("present", elapsed_milliseconds(started_at), compiler_output, target.verified_entry) + elsif compiler_output.includes?(ABSENT_MARKER) + APIProbeResult.new("absent", elapsed_milliseconds(started_at), compiler_output) + else + APIProbeResult.new("unavailable", elapsed_milliseconds(started_at), compiler_output) + end + rescue ex : Exception + unavailable(started_at, ex.message || "Could not run the Crystal API probe") + ensure + FileUtils.rm_rf(probe_root) if probe_root + end + + private def safe_target?(type_name : String, method_name : String) : Bool + safe_type_name?(type_name) && method_name.matches?(/\A[a-zA-Z_][a-zA-Z0-9_]*[!?]?\z/) + end + + private def safe_type_name?(type_name : String) : Bool + type_name.matches?(/\A(?:::)?[A-Z][A-Za-z0-9_]*(?:::[A-Z][A-Za-z0-9_]*)*(?:\([A-Za-z0-9_:, ?|&]+\))?\z/) + end + + private def project_entrypoint : String + manifest_path = File.join(@project_root_path, "shard.yml") + manifest = File.file?(manifest_path) ? ProjectShardManifest.from_yaml(File.read(manifest_path)) : ProjectShardManifest.new + + manifest.targets.values.map(&.main).reject(&.empty?).sort.each do |entrypoint| + return File.expand_path(entrypoint, @project_root_path) if File.file?(File.expand_path(entrypoint, @project_root_path)) + end + + unless manifest.name.empty? + entrypoint = File.join(@project_root_path, "src", "#{manifest.name.gsub('-', '_')}.cr") + return entrypoint if File.file?(entrypoint) + end + + {"config/application.cr", "src/application.cr"}.each do |relative_path| + entrypoint = File.join(@project_root_path, relative_path) + return entrypoint if File.file?(entrypoint) + end + + first_source = Dir.glob(File.join(@project_root_path, "src", "*.cr")).sort.first? + raise APIIndexBuildError.new("No Crystal entrypoint found under #{@project_root_path}/src") unless first_source + + first_source + end + + private def probe_source(require_path : String, target : VerificationTarget) : String + source = String::Builder.new + source << "require " << require_path.inspect << "\n" + source << <<-CRYSTAL + macro amber_lsp_verify_type(type_expression) + {% type_node = parse_type(type_expression.stringify).resolve? %} + {% unless type_node.is_a?(TypeNode) %} + {% raise "#{ABSENT_MARKER}" %} + {% end %} + nil + end + + macro amber_lsp_verify_method(type_expression, method_name, method_scope) + {% type_node = parse_type(type_expression.stringify).resolve? %} + {% unless type_node.is_a?(TypeNode) %} + {% raise "#{ABSENT_MARKER}" %} + {% end %} + {% found = false %} + {% if method_scope == :class %} + {% for method in type_node.class.methods %} + {% if method.name == method_name.id %}{% found = true %}{% end %} + {% end %} + {% for ancestor in type_node.ancestors %} + {% for method in ancestor.class.methods %} + {% if method.name == method_name.id %}{% found = true %}{% end %} + {% end %} + {% end %} + {% for ancestor in type_node.class.ancestors %} + {% for method in ancestor.methods %} + {% if method.name == method_name.id %}{% found = true %}{% end %} + {% end %} + {% end %} + {% for method in type_node.class.all_methods %} + {% if method.name == method_name.id %}{% found = true %}{% end %} + {% end %} + {% else %} + {% for method in type_node.methods %} + {% if method.name == method_name.id %}{% found = true %}{% end %} + {% end %} + {% for ancestor in type_node.ancestors %} + {% for method in ancestor.methods %} + {% if method.name == method_name.id %}{% found = true %}{% end %} + {% end %} + {% end %} + {% for method in type_node.all_methods %} + {% if method.name == method_name.id %}{% found = true %}{% end %} + {% end %} + {% end %} + {% unless found %} + {% raise "#{ABSENT_MARKER}" %} + {% end %} + nil + end + CRYSTAL + source << "\n" + + case target.target_kind + when "type" + source << "amber_lsp_verify_type(#{target.type_name})\n" + when "class", "instance" + method_scope = target.target_kind == "class" ? ":class" : ":instance" + source << "amber_lsp_verify_method(#{target.type_name}, :#{target.method_name}, #{method_scope})\n" + else + raise APIIndexBuildError.new("Unsupported compiler probe target #{target.target_kind}") + end + + source.to_s + end + + private def unavailable(started_at : Time::Instant, reason : String) : APIProbeResult + APIProbeResult.new("unavailable", elapsed_milliseconds(started_at), reason) + end + + private def elapsed_milliseconds(started_at : Time::Instant) : Int64 + (Time.instant - started_at).total_milliseconds.to_i64 + end + + ABSENT_MARKER = "AMBER_LSP_PROBE_ABSENT" + + private struct VerificationTarget + getter type_name : String + getter method_name : String? + getter target_kind : String + getter verified_entry : APIIndexMethod? + + def initialize(@type_name : String, @method_name : String?, @target_kind : String, @verified_entry : APIIndexMethod?) + end + end + end +end From 48660bfc1f2d790952e11575882ba6a8f3da731b Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:18:27 -0400 Subject: [PATCH 23/69] feat(lsp): add amber-lsp lookup command --- spec/amber_lsp/lookup/lookup_command_spec.cr | 156 +++++++++++ src/amber_lsp.cr | 3 + src/amber_lsp/lookup/lookup_command.cr | 270 +++++++++++++++++++ 3 files changed, 429 insertions(+) create mode 100644 spec/amber_lsp/lookup/lookup_command_spec.cr create mode 100644 src/amber_lsp/lookup/lookup_command.cr diff --git a/spec/amber_lsp/lookup/lookup_command_spec.cr b/spec/amber_lsp/lookup/lookup_command_spec.cr new file mode 100644 index 0000000..51d80b8 --- /dev/null +++ b/spec/amber_lsp/lookup/lookup_command_spec.cr @@ -0,0 +1,156 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/lookup_command" + +describe AmberLSP::Lookup::ExtractLookupQueryAtPosition do + it "uses a constant receiver and preserves its method separator" do + source = "CLIProject::User.where(name: \"Ada\")\n" + query = AmberLSP::Lookup::ExtractLookupQueryAtPosition.new(source, 0, 22).perform + + query.should_not be_nil + query.not_nil!.query.should eq("CLIProject::User.where") + end + + it "uses a bare method query for a non-constant receiver" do + source = "user.where(name: \"Ada\")\n" + query = AmberLSP::Lookup::ExtractLookupQueryAtPosition.new(source, 0, 8).perform + + query.should_not be_nil + query.not_nil!.query.should eq("where") + query.not_nil!.receiver.should eq("user") + end + + it "rejects positions without an identifier" do + query = AmberLSP::Lookup::ExtractLookupQueryAtPosition.new("User.\n", 0, 5).perform + + query.should be_nil + end +end + +describe AmberLSP::Lookup::RunLookupCommand do + it "uses exit codes for found, candidates, and unknown index answers" do + with_lookup_cli_project do |project| + cache_root = lookup_cli_cache_root + + found_result = run_lookup_cli(["CLIProject::User#id", "--root", project], cache_root) + candidate_result = run_lookup_cli(["ambiguous_method", "--root", project], cache_root) + unknown_result = run_lookup_cli(["CLIProject::User#missing_api", "--root", project], cache_root) + + found_result[0].should eq(0) + found_result[1].should contain("amber-lsp lookup: found (") + found_result[1].should contain("Int64 | Nil") + candidate_result[0].should eq(3) + candidate_result[1].should contain("amber-lsp lookup: candidates (") + unknown_result[0].should eq(4) + unknown_result[1].should contain("amber-lsp lookup: unknown (") + end + end + + it "uses exit code 5 for a compiler-verified absent method" do + with_lookup_cli_project do |project| + result = run_lookup_cli( + ["CLIProject::User#missing_api", "--root", project, "--verify"], + lookup_cli_cache_root, + ) + + result[0].should eq(5) + result[1].should contain("amber-lsp lookup: absent (") + result[2].should contain("verified absent in") + end + end + + it "uses exit code 2 when the compiler cannot run" do + with_lookup_cli_project do |project| + stdout = IO::Memory.new + stderr = IO::Memory.new + code = AmberLSP::Lookup::RunLookupCommand.new( + ["CLIProject::User#missing_api", "--root", project, "--verify"], + lookup_cli_cache_root, + "crystal-alpha-not-installed", + stdout, + stderr, + ).perform + + code.should eq(2) + stderr.to_s.should contain("amber-lsp lookup failed") + end + end + + it "emits the answer fields and verification duration as JSON" do + with_lookup_cli_project do |project| + result = run_lookup_cli( + ["CLIProject::User#id", "--root", project, "--json", "--verify"], + lookup_cli_cache_root, + ) + json = JSON.parse(result[1]) + + result[0].should eq(0) + json["status"].as_s.should eq("found") + json["freshness"].as_s.should eq("fresh") + json["entries"].as_a.first["name"].as_s.should eq("id") + json["verification_status"].as_s.should eq("present") + json["verification_elapsed_milliseconds"].as_i.should be >= 0 + end + end + + it "turns --at file positions into a constant method query" do + with_lookup_cli_project do |project| + file_path = File.join(project, "src", "hover.cr") + source = "CLIProject::User#id\n" + File.write(file_path, source) + at_location = "#{file_path}:1:19" + result = run_lookup_cli(["--at", at_location, "--root", project], lookup_cli_cache_root) + + result[0].should eq(0) + result[1].should contain("amber-lsp lookup: found (") + result[1].should contain("id") + end + end +end + +private def run_lookup_cli(arguments : Array(String), cache_root : String) : Tuple(Int32, String, String) + stdout = IO::Memory.new + stderr = IO::Memory.new + code = AmberLSP::Lookup::RunLookupCommand.new(arguments, cache_root, "crystal-alpha", stdout, stderr).perform + {code, stdout.to_s, stderr.to_s} +end + +private def lookup_cli_cache_root : String + File.join(Dir.current, ".luna", "lookup-cli-test-cache") +end + +private def with_lookup_cli_project(&) + with_tempdir do |project| + source_path = File.join(project, "src", "cli_lookup_fixture.cr") + Dir.mkdir_p(File.dirname(source_path)) + File.write(File.join(project, "shard.yml"), <<-YAML) + name: cli_lookup_fixture + version: 0.1.0 + targets: + cli_lookup_fixture: + main: src/cli_lookup_fixture.cr + YAML + File.write(source_path, <<-CRYSTAL) + module CLIProject + class Parent + def inherited_api + end + end + + class User < Parent + getter id : Int64? + end + + class One + def ambiguous_method + end + end + + class Two + def ambiguous_method + end + end + end + CRYSTAL + yield project + end +end diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 652ab37..4eb90a9 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -45,10 +45,13 @@ require "./amber_lsp/lookup/build_layered_api_index" require "./amber_lsp/lookup/resolve_api_query" require "./amber_lsp/lookup/answer_api_query" require "./amber_lsp/lookup/verify_api_query" +require "./amber_lsp/lookup/lookup_command" if ARGV.includes?("--version") STDOUT.puts AmberLSP.version_line exit 0 +elsif ARGV.first? == "lookup" + exit AmberLSP::Lookup::RunLookupCommand.new(ARGV[1..].to_a).perform elsif ARGV[0]? == "--check" file_path = ARGV[1]? unless file_path diff --git a/src/amber_lsp/lookup/lookup_command.cr b/src/amber_lsp/lookup/lookup_command.cr new file mode 100644 index 0000000..9f19e83 --- /dev/null +++ b/src/amber_lsp/lookup/lookup_command.cr @@ -0,0 +1,270 @@ +require "json" + +require "./answer_api_query" +require "./build_layered_api_index" +require "./resolve_api_query" +require "./verify_api_query" + +module AmberLSP::Lookup + struct LookupCLIOptions + getter query : String? + getter at_location : String? + getter root_path : String + getter json : Bool + getter verify : Bool + + def initialize( + @query : String?, + @at_location : String?, + @root_path : String, + @json : Bool, + @verify : Bool, + ) + end + end + + struct LookupCLIAnswer + include JSON::Serializable + + getter query : String + getter status : String + getter freshness : String + @[JSON::Field(emit_null: true)] + getter type_summary : APIIndexType? + @[JSON::Field(key: "entries")] + getter list_of_entries : Array(APIIndexMethod) + @[JSON::Field(key: "layers")] + getter list_of_layers : Array(APIIndexLayerState) + @[JSON::Field(emit_null: true)] + getter verification_status : String? + @[JSON::Field(emit_null: true)] + getter verification_elapsed_milliseconds : Int64? + @[JSON::Field(emit_null: true)] + getter failure_reason : String? + + def initialize( + @query : String, + @status : String, + @freshness : String, + @type_summary : APIIndexType?, + @list_of_entries : Array(APIIndexMethod), + @list_of_layers : Array(APIIndexLayerState), + @verification_status : String?, + @verification_elapsed_milliseconds : Int64?, + @failure_reason : String?, + ) + end + end + + struct LookupAtQuery + getter query : String + getter token : String + getter receiver : String? + + def initialize(@query : String, @token : String, @receiver : String?) + end + end + + class ExtractLookupQueryAtPosition + def initialize(@source : String, @line_number : Int32, @column_number : Int32) + end + + def perform : LookupAtQuery? + return nil if @line_number < 0 || @column_number < 0 + + line = @source.lines[@line_number]?.try(&.rstrip("\r\n")) + return nil unless line + + column = Math.min(@column_number, line.bytesize) + before_cursor = line.byte_slice(0, column) + after_cursor = line.byte_slice(column, line.bytesize - column) + left_token = before_cursor.match(/[A-Za-z0-9_!?]+\z/).try(&.[0]) || "" + right_token = after_cursor.match(/\A[A-Za-z0-9_!?]+/).try(&.[0]) || "" + token = left_token + right_token + return nil if token.empty? + + receiver_prefix = before_cursor.byte_slice(0, before_cursor.bytesize - left_token.bytesize) + receiver_match = receiver_prefix.match(/(.+)([.#])\s*\z/) + return LookupAtQuery.new(token, token, nil) unless receiver_match + + receiver = receiver_match[1].strip + separator = receiver_match[2] + if constant_receiver?(receiver) + LookupAtQuery.new("#{receiver}#{separator}#{token}", token, receiver) + else + LookupAtQuery.new(token, token, receiver) + end + end + + private def constant_receiver?(receiver : String) : Bool + receiver.matches?(/\A(?:::)?[A-Z][A-Za-z0-9_]*(?:::[A-Z][A-Za-z0-9_]*)*(?:\([A-Za-z0-9_:, ?|&]+\))?\z/) + end + end + + class RunLookupCommand + def initialize( + @arguments : Array(String), + @cache_root : String = APIIndexCache.default_root, + @compiler_command : String = "crystal-alpha", + @stdout : IO = STDOUT, + @stderr : IO = STDERR, + ) + end + + def perform : Int32 + options = parse_options + query = lookup_query(options) + cached_layers = BuildLayeredAPIIndex.new( + options.root_path, + @cache_root, + {} of String => Array(String), + @compiler_command, + ).perform + list_of_index_layers = cached_layers.compact_map(&.layer) + resolution = ResolveAPIQuery.new(query, list_of_index_layers).perform + answer = AnswerAPIQuery.new(query, resolution, cached_layers).perform + probe = options.verify ? VerifyAPIQuery.new(options.root_path, query, resolution, @compiler_command).perform : nil + cli_answer = make_cli_answer(answer, probe) + + if options.json + @stdout.puts cli_answer.to_json + else + print_plain_answer(cli_answer, probe) + end + + if probe && probe.status == "unavailable" + @stderr.puts("amber-lsp lookup: verification unavailable: #{probe.output}") + return 2 + end + + exit_code_for(cli_answer.status) + rescue ex : Exception + @stderr.puts("amber-lsp lookup failed: #{ex.message || "unknown error"}") + 2 + end + + private def parse_options : LookupCLIOptions + query = nil.as(String?) + at_location = nil.as(String?) + root_path = Dir.current + json = false + verify = false + index = 0 + + while index < @arguments.size + argument = @arguments[index] + case argument + when "--root" + index += 1 + root_path = @arguments[index]? || raise ArgumentError.new("--root requires a directory") + when "--at" + index += 1 + at_location = @arguments[index]? || raise ArgumentError.new("--at requires FILE:LINE:COL") + when "--json" + json = true + when "--verify" + verify = true + else + if argument.starts_with?("-") + raise ArgumentError.new("Unknown lookup option #{argument}") + end + raise ArgumentError.new("Only one lookup query is supported") if query + query = argument + end + index += 1 + end + + raise ArgumentError.new("lookup requires a query or --at FILE:LINE:COL") if !query && !at_location + raise ArgumentError.new("Use either a query or --at, not both") if query && at_location + + LookupCLIOptions.new(query, at_location, File.expand_path(root_path), json, verify) + end + + private def lookup_query(options : LookupCLIOptions) : String + return options.query.not_nil! if options.query + + at_location = options.at_location.not_nil! + match = at_location.match(/\A(.+):(\d+):(\d+)\z/) + raise ArgumentError.new("--at requires FILE:LINE:COL") unless match + + file_path = File.expand_path(match[1]) + line_number = match[2].to_i - 1 + column_number = match[3].to_i - 1 + raise ArgumentError.new("--at line and column numbers start at 1") if line_number < 0 || column_number < 0 + + source = File.read(file_path) + extracted = ExtractLookupQueryAtPosition.new(source, line_number, column_number).perform + raise ArgumentError.new("No Crystal identifier at #{at_location}") unless extracted + + extracted.query + end + + private def make_cli_answer(answer : LookupAnswer, probe : APIProbeResult?) : LookupCLIAnswer + status = answer.status + list_of_entries = answer.list_of_entries + verification_status = probe.try(&.status) + failure_reason = nil.as(String?) + + if probe + case probe.status + when "absent" + status = "absent" + when "present" + if verified_entry = probe.verified_entry + list_of_entries = [verified_entry] + status = "found" + elsif answer.status == "unknown" + status = "present" + end + when "unavailable" + failure_reason = probe.output + end + end + + LookupCLIAnswer.new( + answer.query, + status, + answer.freshness, + answer.type_summary, + list_of_entries, + answer.list_of_layers, + verification_status, + probe.try(&.elapsed_milliseconds), + failure_reason, + ) + end + + private def print_plain_answer(answer : LookupCLIAnswer, probe : APIProbeResult?) : Nil + @stdout.puts("amber-lsp lookup: #{answer.status} (#{answer.freshness})") + if type_summary = answer.type_summary + @stdout.puts("#{type_summary.name} (#{type_summary.kind}) — #{type_summary.location_path}:#{type_summary.location_line} [#{type_summary.source_layer}]") + end + + answer.list_of_entries.each do |entry| + argument_string = entry.args_string.starts_with?('(') ? entry.args_string : "(#{entry.args_string})" + return_type = entry.return_type.gsub("::Nil", "Nil") + @stdout.puts("#{entry.owner}##{entry.name}#{argument_string} : #{return_type} — #{entry.source_path}:#{entry.source_line} [#{entry.source_layer}]") + @stdout.puts(" #{entry.doc_line}") if entry.doc_line + end + + if probe + @stderr.puts("amber-lsp lookup: verified #{probe.status} in #{probe.elapsed_milliseconds} ms") + end + end + + private def exit_code_for(status : String) : Int32 + case status + when "found", "present" + 0 + when "candidates" + 3 + when "unknown" + 4 + when "absent" + 5 + else + 2 + end + end + end +end From 7041a36da8f7dc673311388954f205b1aa10b660 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:21:27 -0400 Subject: [PATCH 24/69] feat(lsp): load versioned Crystal API cards --- spec/amber_lsp/lookup/api_cards_spec.cr | 85 ++++++ .../lookup/build_layered_api_index_spec.cr | 12 + spec/amber_lsp/lookup/lookup_command_spec.cr | 7 + .../cards/library/.amber-lsp/api/example.yml | 12 + .../.amber-lsp/api/fixture_project.yml | 12 + src/amber_lsp/lookup/api_cards.cr | 259 ++++++++++++++++++ .../lookup/build_layered_api_index.cr | 11 +- src/amber_lsp/lookup/lookup_command.cr | 37 ++- 8 files changed, 429 insertions(+), 6 deletions(-) create mode 100644 spec/amber_lsp/lookup/api_cards_spec.cr create mode 100644 spec/fixtures/api_lookup/cards/library/.amber-lsp/api/example.yml create mode 100644 spec/fixtures/api_lookup/cards/project/.amber-lsp/api/fixture_project.yml create mode 100644 src/amber_lsp/lookup/api_cards.cr diff --git a/spec/amber_lsp/lookup/api_cards_spec.cr b/spec/amber_lsp/lookup/api_cards_spec.cr new file mode 100644 index 0000000..950dc23 --- /dev/null +++ b/spec/amber_lsp/lookup/api_cards_spec.cr @@ -0,0 +1,85 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/api_cards" + +describe AmberLSP::Lookup::APIVersionRequirement do + it "matches exact, comparator, and pessimistic semantic version requirements" do + AmberLSP::Lookup::APIVersionRequirement.new("1.2.3").matches?("1.2.3").should be_true + AmberLSP::Lookup::APIVersionRequirement.new(">= 1.0, < 2.0.0").matches?("1.5.0").should be_true + AmberLSP::Lookup::APIVersionRequirement.new("~> 0.3.0").matches?("0.3.9").should be_true + AmberLSP::Lookup::APIVersionRequirement.new("~> 0.3.0").matches?("0.4.0").should be_false + end +end + +describe AmberLSP::Lookup::LoadAPICards do + it "loads the project card and locked library cards with notes and error hints" do + with_api_card_project do |project| + cards = AmberLSP::Lookup::LoadAPICards.new(project).perform + + cards.list_of_errors.should be_empty + cards.list_of_cards.map(&.card.library).should eq(["fixture_project", "example"]) + cards.docs_flags_by_library.should eq({ + "fixture_project" => ["fixture_project_docs"], + "example" => ["example_docs"], + }) + cards.matching_notes("FixtureAPI::User#id").map(&.text).should contain( + "The ID getter is generated from the persisted table column." + ) + cards.matching_notes("Example::Widget#id").map(&.text).should contain( + "This getter follows the library's primary key." + ) + cards.matching_error_hints("undefined method 'lookup' for Example::Widget").map(&.example).should contain( + "Example::Widget.lookup(\"widget-1\")" + ) + end + end + + it "applies the owner repository card against the project's own version" do + with_api_card_project do |project| + File.write(File.join(project, "shard.yml"), "name: fixture_project\nversion: 1.5.0\n") + cards = AmberLSP::Lookup::LoadAPICards.new(project).perform + + cards.list_of_cards.map(&.card.library).should contain("fixture_project") + end + end + + it "ignores cards whose applies_to range does not match the locked version" do + with_api_card_project do |project| + File.write(File.join(project, "shard.lock"), "shards:\n example:\n version: 0.4.0\n") + cards = AmberLSP::Lookup::LoadAPICards.new(project).perform + + cards.list_of_cards.map(&.card.library).should_not contain("example") + cards.list_of_errors.should be_empty + end + end + + it "rejects unsupported card versions and malformed error regular expressions" do + with_api_card_project do |project| + card_path = File.join(project, "lib", "example", ".amber-lsp", "api", "example.yml") + card = File.read(card_path).gsub("card_version: 1", "card_version: 2") + File.write(card_path, card) + + cards = AmberLSP::Lookup::LoadAPICards.new(project).perform + + cards.list_of_cards.map(&.card.library).should_not contain("example") + cards.list_of_errors.first.should contain("card_version must be 1") + + invalid_pattern_card = File.read(card_path).gsub("card_version: 2", "card_version: 1").gsub("pattern: \"undefined method 'lookup'\"", "pattern: \"[\"") + File.write(card_path, invalid_pattern_card) + invalid_pattern_result = AmberLSP::Lookup::LoadAPICards.new(project).perform + invalid_pattern_result.list_of_cards.map(&.card.library).should_not contain("example") + invalid_pattern_result.list_of_errors.first.should contain(card_path) + end + end +end + +private def with_api_card_project(&) + with_tempdir do |project| + fixture_root = File.join(Dir.current, "spec", "fixtures", "api_lookup", "cards") + File.write(File.join(project, "shard.yml"), "name: fixture_project\nversion: 1.5.0\n") + FileUtils.cp_r(File.join(fixture_root, "project", ".amber-lsp"), project) + Dir.mkdir_p(File.join(project, "lib", "example")) + FileUtils.cp_r(File.join(fixture_root, "library", ".amber-lsp"), File.join(project, "lib", "example")) + File.write(File.join(project, "shard.lock"), "shards:\n example:\n version: 0.3.5\n") + yield project + end +end diff --git a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr index 2bf5c1d..ea0af3d 100644 --- a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr +++ b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr @@ -8,6 +8,8 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do cache_root = File.join(root, "cache") Dir.mkdir_p(File.join(project_root, "src")) Dir.mkdir_p(File.join(project_root, "lib", "tiny", "src")) + Dir.mkdir_p(File.join(project_root, ".amber-lsp", "api")) + Dir.mkdir_p(File.join(project_root, "lib", "tiny", ".amber-lsp", "api")) File.write( File.join(project_root, "shard.yml"), "name: lookup_fixture\nversion: 1.0.0\ndependencies:\n tiny:\n github: example/tiny\ntargets:\n app:\n main: src/lookup_fixture.cr\n", @@ -28,6 +30,14 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do File.join(project_root, "src", "lookup_fixture.cr"), "require \"tiny\"\nclass FixtureUser\n def id : Int64?\n 1_i64\n end\nend\n", ) + File.write( + File.join(project_root, ".amber-lsp", "api", "lookup_fixture.yml"), + "card_version: 1\nlibrary: lookup_fixture\napplies_to: 1.0.0\ndocs_flags:\n - fixture_docs\n", + ) + File.write( + File.join(project_root, "lib", "tiny", ".amber-lsp", "api", "tiny.yml"), + "card_version: 1\nlibrary: tiny\napplies_to: '~> 1.2.0'\ndocs_flags:\n - tiny_docs\n", + ) builder = AmberLSP::Lookup::BuildLayeredAPIIndex.new(project_root, cache_root) first_build = builder.perform @@ -42,6 +52,8 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do project_layer.not_nil!.freshness.should eq("fresh") library_layer.not_nil!.freshness.should eq("fresh") standard_library_layer.not_nil!.freshness.should eq("fresh") + project_layer.not_nil!.layer.not_nil!.docs_flags.should eq(["fixture_docs"]) + library_layer.not_nil!.layer.not_nil!.docs_flags.should eq(["tiny_docs"]) fixture_user = project_layer.not_nil!.layer.not_nil!.list_of_types.find { |type| type.name == "FixtureUser" } fixture_user.should_not be_nil diff --git a/spec/amber_lsp/lookup/lookup_command_spec.cr b/spec/amber_lsp/lookup/lookup_command_spec.cr index 51d80b8..89d090a 100644 --- a/spec/amber_lsp/lookup/lookup_command_spec.cr +++ b/spec/amber_lsp/lookup/lookup_command_spec.cr @@ -38,6 +38,7 @@ describe AmberLSP::Lookup::RunLookupCommand do found_result[0].should eq(0) found_result[1].should contain("amber-lsp lookup: found (") found_result[1].should contain("Int64 | Nil") + found_result[1].should contain("Note [CLIProject::User#id]") candidate_result[0].should eq(3) candidate_result[1].should contain("amber-lsp lookup: candidates (") unknown_result[0].should eq(4) @@ -89,6 +90,7 @@ describe AmberLSP::Lookup::RunLookupCommand do json["entries"].as_a.first["name"].as_s.should eq("id") json["verification_status"].as_s.should eq("present") json["verification_elapsed_milliseconds"].as_i.should be >= 0 + json["card_notes"].as_a.size.should eq(1) end end @@ -122,6 +124,7 @@ private def with_lookup_cli_project(&) with_tempdir do |project| source_path = File.join(project, "src", "cli_lookup_fixture.cr") Dir.mkdir_p(File.dirname(source_path)) + Dir.mkdir_p(File.join(project, ".amber-lsp", "api")) File.write(File.join(project, "shard.yml"), <<-YAML) name: cli_lookup_fixture version: 0.1.0 @@ -151,6 +154,10 @@ private def with_lookup_cli_project(&) end end CRYSTAL + File.write( + File.join(project, ".amber-lsp", "api", "cli_lookup_fixture.yml"), + "card_version: 1\nlibrary: cli_lookup_fixture\napplies_to: 0.1.0\nnotes:\n - symbol: CLIProject::User#id\n text: The ID is a nullable persisted column.\n", + ) yield project end end diff --git a/spec/fixtures/api_lookup/cards/library/.amber-lsp/api/example.yml b/spec/fixtures/api_lookup/cards/library/.amber-lsp/api/example.yml new file mode 100644 index 0000000..5530e42 --- /dev/null +++ b/spec/fixtures/api_lookup/cards/library/.amber-lsp/api/example.yml @@ -0,0 +1,12 @@ +card_version: 1 +library: example +applies_to: "~> 0.3.0" +docs_flags: + - example_docs +notes: + - symbol: Example::Widget#id + text: This getter follows the library's primary key. +error_hints: + - pattern: "undefined method 'lookup'" + hint: Use the lookup helper defined on the widget's query module. + example: Example::Widget.lookup("widget-1") diff --git a/spec/fixtures/api_lookup/cards/project/.amber-lsp/api/fixture_project.yml b/spec/fixtures/api_lookup/cards/project/.amber-lsp/api/fixture_project.yml new file mode 100644 index 0000000..b583a45 --- /dev/null +++ b/spec/fixtures/api_lookup/cards/project/.amber-lsp/api/fixture_project.yml @@ -0,0 +1,12 @@ +card_version: 1 +library: fixture_project +applies_to: ">= 1.0.0, < 2.0.0" +docs_flags: + - fixture_project_docs +notes: + - symbol: FixtureAPI::User#id + text: The ID getter is generated from the persisted table column. +error_hints: + - pattern: "undefined method 'save'" + hint: Persist the model with its supported save operation. + example: user.save diff --git a/src/amber_lsp/lookup/api_cards.cr b/src/amber_lsp/lookup/api_cards.cr new file mode 100644 index 0000000..729cbae --- /dev/null +++ b/src/amber_lsp/lookup/api_cards.cr @@ -0,0 +1,259 @@ +require "semantic_version" +require "json" +require "yaml" + +require "./source_models" + +module AmberLSP::Lookup + struct APICardNote + include YAML::Serializable + include JSON::Serializable + + getter symbol : String + getter text : String + end + + struct APICardErrorHint + include YAML::Serializable + include JSON::Serializable + + getter pattern : String + getter hint : String + getter example : String + end + + struct APICard + include YAML::Serializable + + getter card_version : Int32 + getter library : String + getter applies_to : String + getter docs_flags : Array(String) = [] of String + getter notes : Array(APICardNote) = [] of APICardNote + getter error_hints : Array(APICardErrorHint) = [] of APICardErrorHint + end + + struct LoadedAPICard + getter card : APICard + getter source_path : String + getter resolved_version : String + + def initialize(@card : APICard, @source_path : String, @resolved_version : String) + end + end + + struct APICardCollection + getter list_of_cards : Array(LoadedAPICard) + getter list_of_errors : Array(String) + + def initialize(@list_of_cards : Array(LoadedAPICard), @list_of_errors : Array(String)) + end + + def docs_flags_by_library : Hash(String, Array(String)) + list_of_cards = {} of String => Array(String) + @list_of_cards.each do |loaded_card| + list_of_cards[loaded_card.card.library] = loaded_card.card.docs_flags + end + list_of_cards + end + + def matching_notes(query : String) : Array(APICardNote) + query_method_name = query.split(/[.#]/).last? + @list_of_cards.flat_map do |loaded_card| + loaded_card.card.notes.select do |note| + note.symbol == query || note.symbol == query_method_name + end + end + end + + def matching_error_hints(error_text : String) : Array(APICardErrorHint) + @list_of_cards.flat_map do |loaded_card| + loaded_card.card.error_hints.select do |hint| + Regex.new(hint.pattern).matches?(error_text) + end + end + end + end + + class APIVersionRequirement + def initialize(@requirement : String) + end + + def matches?(version_text : String) : Bool + current_version = parse_version(version_text) + return false unless current_version + + requirement = @requirement.strip + return true if requirement.empty? || requirement == "*" + + requirement.split(',').all? do |clause| + matches_clause?(current_version, clause.strip) + end + rescue ex : ArgumentError + false + end + + private def matches_clause?(current_version : SemanticVersion, clause : String) : Bool + if match = clause.match(/\A(~>|>=|<=|!=|==|>|<|=)?\s*(\d+\.\d+(?:\.\d+)?(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?)\z/) + operator = match[1]? || "=" + requested_version = parse_version(match[2]) + return false unless requested_version + + case operator + when "~>" + upper_version = pessimistic_upper_bound(match[2], requested_version) + current_version >= requested_version && current_version < upper_version + when ">=" + current_version >= requested_version + when "<=" + current_version <= requested_version + when ">" + current_version > requested_version + when "<" + current_version < requested_version + when "!=" + current_version != requested_version + else + current_version == requested_version + end + else + false + end + end + + private def parse_version(version_text : String) : SemanticVersion? + normalized = normalize_version(version_text.strip) + SemanticVersion.parse?(normalized) + end + + private def normalize_version(version_text : String) : String + version_without_metadata = version_text.split('+', 2).first || version_text + version_core = version_without_metadata.split('-', 2).first || version_without_metadata + component_count = version_core.split('.').size + suffix = version_text[version_core.bytesize..]? || "" + case component_count + when 1 + "#{version_core}.0.0#{suffix}" + when 2 + "#{version_core}.0#{suffix}" + else + version_text + end + end + + private def pessimistic_upper_bound(version_text : String, version : SemanticVersion) : SemanticVersion + component_count = version_text.split(/[+-]/).first.to_s.split('.').size + if component_count <= 2 + SemanticVersion.new(version.major + 1, 0, 0) + else + SemanticVersion.new(version.major, version.minor + 1, 0) + end + end + end + + class LoadAPICards + def initialize(@project_root_path : String) + @project_root_path = File.expand_path(@project_root_path) + end + + def perform : APICardCollection + project_manifest = read_project_manifest + list_of_targets = [CardSource.new(@project_root_path, project_manifest.name, project_manifest.version)] + list_of_targets.concat(library_card_sources) + + list_of_cards = [] of LoadedAPICard + list_of_errors = [] of String + list_of_targets.each do |target| + next if target.library_name.empty? + + card_paths(target.root_path).each do |path| + begin + card = APICard.from_yaml(File.read(path)) + validate_card(card, target, path) + next unless APIVersionRequirement.new(card.applies_to).matches?(target.resolved_version) + + list_of_cards << LoadedAPICard.new(card, path, target.resolved_version) + rescue ex : YAML::ParseException | IO::Error | ArgumentError + list_of_errors << "#{path}: #{ex.message}" + end + end + end + + duplicate_library_cards(list_of_cards).each do |library_name| + list_of_errors << "More than one API card applies to #{library_name}" + list_of_cards.reject! { |loaded_card| loaded_card.card.library == library_name } + end + + APICardCollection.new(list_of_cards, list_of_errors) + rescue ex : YAML::ParseException | IO::Error + APICardCollection.new([] of LoadedAPICard, [ex.message || "Could not load API cards"]) + end + + private def read_project_manifest : ProjectShardManifest + manifest_path = File.join(@project_root_path, "shard.yml") + return ProjectShardManifest.new unless File.file?(manifest_path) + + ProjectShardManifest.from_yaml(File.read(manifest_path)) + end + + private def library_card_sources : Array(CardSource) + lock_path = File.join(@project_root_path, "shard.lock") + return [] of CardSource unless File.file?(lock_path) + + lockfile = ShardLockfile.from_yaml(File.read(lock_path)) + lockfile.shards.to_a.sort_by(&.first).compact_map do |library_name, locked_shard| + library_root = File.join(@project_root_path, "lib", library_name) + next unless File.directory?(library_root) + + CardSource.new(library_root, library_name, locked_shard.version) + end + end + + private def card_paths(root_path : String) : Array(String) + Dir.glob(File.join(root_path, ".amber-lsp", "api", "*.yml")).sort + end + + private def validate_card(card : APICard, target : CardSource, path : String) : Nil + raise ArgumentError.new("card_version must be 1") unless card.card_version == 1 + raise ArgumentError.new("library must be #{target.library_name}") unless card.library == target.library_name + raise ArgumentError.new("applies_to must be a supported semantic version requirement") unless valid_requirement?(card.applies_to) + + card.docs_flags.each do |flag| + raise ArgumentError.new("invalid docs flag #{flag.inspect}") unless flag.matches?(/\A[A-Za-z_][A-Za-z0-9_]*\z/) + end + + card.notes.each do |note| + raise ArgumentError.new("each note needs a symbol and text") if note.symbol.empty? || note.text.empty? + end + + card.error_hints.each do |hint| + raise ArgumentError.new("each error hint needs a pattern, hint, and example") if hint.pattern.empty? || hint.hint.empty? || hint.example.empty? + Regex.new(hint.pattern) + end + rescue ex : ArgumentError + raise ArgumentError.new("#{path}: #{ex.message}") + end + + private def valid_requirement?(requirement : String) : Bool + return false if requirement.strip.empty? + return true if requirement.strip == "*" + + requirement.split(',').all? do |clause| + clause.strip.matches?(/\A(?:~>|>=|<=|!=|==|>|<|=)?\s*\d+\.\d+(?:\.\d+)?(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?\z/) + end + end + + private def duplicate_library_cards(cards : Array(LoadedAPICard)) : Array(String) + cards.group_by(&.card.library).select { |_, list_of_cards| list_of_cards.size > 1 }.keys + end + + private struct CardSource + getter root_path : String + getter library_name : String + getter resolved_version : String + + def initialize(@root_path : String, @library_name : String, @resolved_version : String) + end + end + end +end diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr index 30ac288..abd1633 100644 --- a/src/amber_lsp/lookup/build_layered_api_index.cr +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -3,6 +3,7 @@ require "process" require "yaml" require "./build_crystal_docs_json" +require "./api_cards" require "./index_cache" require "./normalize_crystal_docs" require "./source_models" @@ -58,11 +59,13 @@ module AmberLSP::Lookup def perform : Array(CachedAPIIndexLayer) manifest = read_project_manifest + card_flags = LoadAPICards.new(@root_path).perform.docs_flags_by_library + list_of_docs_flags = card_flags.merge(@list_of_docs_flags_by_library) identity = DetectCrystalAlpha.new(@compiler_command).perform cache = APIIndexCache.new(@cache_root, @root_path) list_of_layers = [] of CachedAPIIndexLayer project_name = manifest.name.empty? ? File.basename(@root_path) : manifest.name - project_flags = docs_flags_for(project_name) + project_flags = docs_flags_for(project_name, list_of_docs_flags) project_key = CalculateProjectLayerKey.new(@root_path, project_flags, identity.version).perform project_entrypoint = find_project_entrypoint(manifest) @@ -125,7 +128,7 @@ module AmberLSP::Lookup next end - docs_flags = docs_flags_for(library_name) + docs_flags = docs_flags_for(library_name, list_of_docs_flags) layer_key = CalculateLibraryLayerKey.new(library_name, locked_shard.version, docs_flags, identity.version).perform environment_overrides = {"CRYSTAL_PATH" => project_crystal_path || ""} @@ -200,8 +203,8 @@ module AmberLSP::Lookup Dir.glob(File.join(source_root, "*.cr")).sort.first? || Dir.glob(File.join(source_root, "**", "*.cr")).sort.first? end - private def docs_flags_for(library_name : String) : Array(String) - @list_of_docs_flags_by_library[library_name]?.try(&.sort) || [] of String + private def docs_flags_for(library_name : String, list_of_docs_flags : Hash(String, Array(String))) : Array(String) + list_of_docs_flags[library_name]?.try(&.sort) || [] of String end private def build_or_load_layer( diff --git a/src/amber_lsp/lookup/lookup_command.cr b/src/amber_lsp/lookup/lookup_command.cr index 9f19e83..ce43e4a 100644 --- a/src/amber_lsp/lookup/lookup_command.cr +++ b/src/amber_lsp/lookup/lookup_command.cr @@ -41,6 +41,12 @@ module AmberLSP::Lookup getter verification_elapsed_milliseconds : Int64? @[JSON::Field(emit_null: true)] getter failure_reason : String? + @[JSON::Field(key: "card_notes")] + getter list_of_card_notes : Array(APICardNote) + @[JSON::Field(key: "error_hints")] + getter list_of_error_hints : Array(APICardErrorHint) + @[JSON::Field(key: "api_card_errors")] + getter list_of_api_card_errors : Array(String) def initialize( @query : String, @@ -52,6 +58,9 @@ module AmberLSP::Lookup @verification_status : String?, @verification_elapsed_milliseconds : Int64?, @failure_reason : String?, + @list_of_card_notes : Array(APICardNote) = [] of APICardNote, + @list_of_error_hints : Array(APICardErrorHint) = [] of APICardErrorHint, + @list_of_api_card_errors : Array(String) = [] of String, ) end end @@ -114,6 +123,7 @@ module AmberLSP::Lookup def perform : Int32 options = parse_options query = lookup_query(options) + card_collection = LoadAPICards.new(options.root_path).perform cached_layers = BuildLayeredAPIIndex.new( options.root_path, @cache_root, @@ -124,7 +134,9 @@ module AmberLSP::Lookup resolution = ResolveAPIQuery.new(query, list_of_index_layers).perform answer = AnswerAPIQuery.new(query, resolution, cached_layers).perform probe = options.verify ? VerifyAPIQuery.new(options.root_path, query, resolution, @compiler_command).perform : nil - cli_answer = make_cli_answer(answer, probe) + note_list = card_collection.matching_notes(query) + hint_list = probe.try(&.output).try { |output| card_collection.matching_error_hints(output) } || [] of APICardErrorHint + cli_answer = make_cli_answer(answer, probe, note_list, hint_list, card_collection.list_of_errors) if options.json @stdout.puts cli_answer.to_json @@ -199,7 +211,13 @@ module AmberLSP::Lookup extracted.query end - private def make_cli_answer(answer : LookupAnswer, probe : APIProbeResult?) : LookupCLIAnswer + private def make_cli_answer( + answer : LookupAnswer, + probe : APIProbeResult?, + list_of_card_notes : Array(APICardNote), + list_of_error_hints : Array(APICardErrorHint), + list_of_api_card_errors : Array(String), + ) : LookupCLIAnswer status = answer.status list_of_entries = answer.list_of_entries verification_status = probe.try(&.status) @@ -231,6 +249,9 @@ module AmberLSP::Lookup verification_status, probe.try(&.elapsed_milliseconds), failure_reason, + list_of_card_notes, + list_of_error_hints, + list_of_api_card_errors, ) end @@ -247,6 +268,18 @@ module AmberLSP::Lookup @stdout.puts(" #{entry.doc_line}") if entry.doc_line end + answer.list_of_card_notes.each do |note| + @stdout.puts(" Note [#{note.symbol}]: #{note.text}") + end + answer.list_of_error_hints.each do |hint| + @stdout.puts(" Hint: #{hint.hint}") + @stdout.puts(" Example: #{hint.example}") + end + + answer.list_of_api_card_errors.each do |error| + @stderr.puts("amber-lsp lookup: API card warning: #{error}") + end + if probe @stderr.puts("amber-lsp lookup: verified #{probe.status} in #{probe.elapsed_milliseconds} ms") end From 0288d97dd320840e04c8e6bdbd138eca26885267 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:50:45 -0400 Subject: [PATCH 25/69] feat(lsp): add API lookup LSP methods --- scripts/lsp_smoke.rb | 191 ++++++++++--- spec/amber_lsp/controller_spec.cr | 5 +- .../lookup/api_index_service_spec.cr | 56 ++++ spec/amber_lsp/lookup/index_cache_spec.cr | 20 ++ spec/amber_lsp/lookup/lookup_command_spec.cr | 16 ++ .../lookup/lsp_lookup_controller_spec.cr | 140 ++++++++++ src/amber_lsp/controller.cr | 264 +++++++++++++++++- src/amber_lsp/lookup/api_index_service.cr | 87 ++++++ .../lookup/build_layered_api_index.cr | 10 +- .../extract_lookup_query_at_position.cr | 63 +++++ src/amber_lsp/lookup/index_cache.cr | 92 ++++++ src/amber_lsp/lookup/lookup_command.cr | 80 ++---- src/amber_lsp/lookup/lsp_models.cr | 137 +++++++++ 13 files changed, 1044 insertions(+), 117 deletions(-) create mode 100644 spec/amber_lsp/lookup/api_index_service_spec.cr create mode 100644 spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr create mode 100644 src/amber_lsp/lookup/api_index_service.cr create mode 100644 src/amber_lsp/lookup/extract_lookup_query_at_position.cr create mode 100644 src/amber_lsp/lookup/lsp_models.cr diff --git a/scripts/lsp_smoke.rb b/scripts/lsp_smoke.rb index e20978a..7f088ca 100755 --- a/scripts/lsp_smoke.rb +++ b/scripts/lsp_smoke.rb @@ -2,21 +2,21 @@ # frozen_string_literal: true # lsp_smoke.rb — live-fire proof that the built `amber-lsp` binary really -# speaks LSP over stdio and really publishes diagnostics. +# speaks LSP over stdio, publishes diagnostics, and serves API lookups. # # The Crystal specs under spec/amber_lsp/ exercise the server class in-process # with an IO::Memory pair. That proves the code, not the binary: it cannot # catch a stale binary on disk, a broken build, or a server that hangs instead # of answering. This script spawns the ACTUAL executable, drives a real framed -# stdio session against a throwaway Amber-shaped project, and asserts on the -# `textDocument/publishDiagnostics` notifications that come back. +# stdio session against a throwaway Amber-shaped project, and checks diagnostics +# plus workspace/symbol, hover, and definition responses. # # scripts/lsp_smoke.rb # uses ./bin/amber-lsp # scripts/lsp_smoke.rb --server /path/to/amber-lsp # scripts/lsp_smoke.rb --timeout 20 --keep # keep the fixture project # -# Exit 0 = the violating fixture produced >= 1 diagnostic, the clean fixture -# produced exactly 0, and an unsaved didChange produced its expected diagnostic. +# Exit 0 = the diagnostic fixtures pass and the binary resolves the fixture +# symbol through workspace/symbol, hover, and definition. # Exit 1 = live-fire expectations not met. Exit 2 = could not run at all (no # binary, handshake failure, timeout). A timeout is never reported as a pass — # "I could not measure it" is not "it is clean". @@ -106,8 +106,21 @@ def build_fixture(dir) dependencies: amber: github: amberframework/amber + targets: + lsp_smoke_lookup: + main: src/lsp_smoke_lookup.cr YAML + File.write(File.join(dir, "src", "lsp_smoke_lookup.cr"), <<~CRYSTAL) + module SmokeLookup + class User + # Returns the user's primary key. + getter id : Int64? + end + end + # Hover target: SmokeLookup::User#id + CRYSTAL + # Violating: class name does not end in Controller (amber/controller-naming, # error severity) and the action never renders (amber/action-return-type). File.write(File.join(dir, "src", "controllers", "users_controller.cr"), <<~CRYSTAL) @@ -142,52 +155,83 @@ def session(server_bin, dir, timeout) clean_content = File.read(good_file_path) unsaved_content = clean_content.sub("PostsController", "PostsHandler") raise "could not prepare unsaved didChange content" if unsaved_content == clean_content - - messages = [ - frame("jsonrpc" => "2.0", "id" => 1, "method" => "initialize", - "params" => { "rootUri" => root_uri, "capabilities" => {} }), - frame("jsonrpc" => "2.0", "method" => "initialized", "params" => {}), - frame("jsonrpc" => "2.0", "method" => "textDocument/didOpen", - "params" => { "textDocument" => { - "uri" => bad_uri, "languageId" => "crystal", "version" => 1, - "text" => File.read(File.join(dir, "src", "controllers", "users_controller.cr")) - } }), - frame("jsonrpc" => "2.0", "method" => "textDocument/didOpen", - "params" => { "textDocument" => { - "uri" => good_uri, "languageId" => "crystal", "version" => 1, - "text" => clean_content - } }), - frame("jsonrpc" => "2.0", "method" => "textDocument/didChange", - "params" => { "textDocument" => { "uri" => good_uri, "version" => 2 }, - "contentChanges" => [{ "text" => unsaved_content }] }), - frame("jsonrpc" => "2.0", "id" => 2, "method" => "shutdown"), - frame("jsonrpc" => "2.0", "method" => "exit") - ] - published = Hash.new { |hash, uri| hash[uri] = [] } - published_count = 0 - initialized = false + status_messages = [] + lookup_uri = "file://#{File.join(dir, "src", "lsp_smoke_lookup.cr")}" io = IO.popen([server_bin], "r+", err: File::NULL) begin io.binmode - io.write(messages.join) - io.flush - reader = FrameReader.new(io) deadline = Time.now + timeout - while published_count < 3 || !initialized - raw = reader.read_frame(deadline) - break if raw.nil? - - msg = JSON.parse(raw) - initialized = true if msg["id"] == 1 && msg.key?("result") - next unless msg["method"] == "textDocument/publishDiagnostics" + initialize_response = request( + io, reader, 1, "initialize", { "rootUri" => root_uri, "capabilities" => {} }, + deadline, published, status_messages + ) + notify(io, "initialized", {}) + notify(io, "textDocument/didOpen", { + "textDocument" => { + "uri" => bad_uri, "languageId" => "crystal", "version" => 1, + "text" => File.read(File.join(dir, "src", "controllers", "users_controller.cr")) + } + }) + notify(io, "textDocument/didOpen", { + "textDocument" => { + "uri" => good_uri, "languageId" => "crystal", "version" => 1, + "text" => clean_content + } + }) + notify(io, "textDocument/didChange", { + "textDocument" => { "uri" => good_uri, "version" => 2 }, + "contentChanges" => [{ "text" => unsaved_content }] + }) + + workspace_symbol_response = nil + request_id = 3 + while Time.now < deadline + workspace_symbol_response = request( + io, reader, request_id, "workspace/symbol", { "query" => "SmokeLookup::User#id" }, + deadline, published, status_messages + ) + result_symbols = workspace_symbol_response["result"] + unless result_symbols.is_a?(Array) + raise "workspace/symbol returned #{JSON.pretty_generate(workspace_symbol_response)}" + end + found_symbol = result_symbols.any? do |symbol| + symbol["name"].include?("SmokeLookup::User#id") + end + break if found_symbol - published[msg["params"]["uri"]] << msg - published_count += 1 + request_id += 1 + sleep 0.05 end + + lookup_source = File.read(File.join(dir, "src", "lsp_smoke_lookup.cr")) + lookup_line_number = lookup_source.lines.index { |line| line.include?("SmokeLookup::User#id") } + raise "could not locate hover target in fixture" if lookup_line_number.nil? + lookup_line = lookup_source.lines[lookup_line_number] + lookup_character = lookup_line.index("#id") + 2 + + hover_response = request( + io, reader, 40, "textDocument/hover", + { "textDocument" => { "uri" => lookup_uri }, "position" => { "line" => lookup_line_number, "character" => lookup_character } }, + deadline, published, status_messages + ) + definition_response = request( + io, reader, 41, "textDocument/definition", + { "textDocument" => { "uri" => lookup_uri }, "position" => { "line" => lookup_line_number, "character" => lookup_character } }, + deadline, published, status_messages + ) + shutdown_response = request(io, reader, 2, "shutdown", {}, deadline, published, status_messages) + notify(io, "exit", {}) + responses = { + initialize: initialize_response, + workspace_symbol: workspace_symbol_response, + hover: hover_response, + definition: definition_response, + shutdown: shutdown_response + } ensure begin io.close @@ -197,8 +241,13 @@ def session(server_bin, dir, timeout) end { - initialized: initialized, + initialized: responses[:initialize] && responses[:initialize]["result"].is_a?(Hash), published: published, + lookup_uri: lookup_uri, + workspace_symbols: responses[:workspace_symbol] && responses[:workspace_symbol]["result"], + hover: responses[:hover], + definition: responses[:definition], + status_messages: status_messages, bad_uri: bad_uri, good_uri: good_uri, good_file_path: good_file_path, @@ -206,6 +255,29 @@ def session(server_bin, dir, timeout) } end + def notify(io, method, params) + io.write(frame("jsonrpc" => "2.0", "method" => method, "params" => params)) + io.flush + end + + def request(io, reader, id, method, params, deadline, published, status_messages) + io.write(frame("jsonrpc" => "2.0", "id" => id, "method" => method, "params" => params)) + io.flush + + loop do + raw = reader.read_frame(deadline) + raise "timed out waiting for #{method}" if raw.nil? + + message = JSON.parse(raw) + if message["method"] == "textDocument/publishDiagnostics" + published[message["params"]["uri"]] << message + elsif message["method"] == "window/logMessage" + status_messages << message["params"]["message"] + end + return message if message["id"] == id + end + end + def describe(diagnostics) diagnostics.map do |d| line = d["range"]["start"]["line"].to_i + 1 @@ -286,6 +358,23 @@ def main(argv) puts describe(changed_diags) puts + workspace_symbols = result[:workspace_symbols] || [] + lookup_symbol = workspace_symbols.find do |symbol| + symbol["name"].include?("SmokeLookup::User#id") + end + hover = result[:hover] && result[:hover]["result"] + hover_value = hover && hover.dig("contents", "value") + definition = result[:definition] && result[:definition]["result"] + definition_location = definition.is_a?(Array) ? definition.first : nil + + puts "--- workspace/symbol: SmokeLookup::User#id ---" + puts JSON.pretty_generate(lookup_symbol) if lookup_symbol + puts "--- textDocument/hover: SmokeLookup::User#id ---" + puts hover_value if hover_value + puts "--- textDocument/definition: SmokeLookup::User#id ---" + puts JSON.pretty_generate(definition_location) if definition_location + puts + ok = true if bad_diags.empty? warn "FAIL: violating fixture produced 0 diagnostics (expected >= 1)." @@ -303,9 +392,25 @@ def main(argv) warn "FAIL: didChange altered the clean fixture on disk." ok = false end + unless lookup_symbol && lookup_symbol.dig("data", "source_layer") == "lsp_smoke_app" + warn "FAIL: workspace/symbol did not return the fixture API entry from the project layer." + ok = false + end + unless hover_value && hover_value.include?("SmokeLookup::User#id() : Int64 | Nil") && + hover_value.include?("Returns the user's primary key.") && + hover_value.include?("Layer: lsp_smoke_app") && + hover_value.include?("Freshness: unavailable") + warn "FAIL: hover did not return the indexed signature, documentation, layer, and freshness." + ok = false + end + unless definition_location && definition_location["uri"] == result[:lookup_uri] && + definition_location.dig("range", "start", "line") == 3 + warn "FAIL: definition did not return the fixture getter location." + ok = false + end if ok - puts "LIVE-FIRE OK: violating=#{bad_diags.size} diagnostic(s), clean=0, didChange=#{changed_diags.size} unsaved diagnostic(s), disk=unchanged." + puts "LIVE-FIRE OK: violating=#{bad_diags.size} diagnostic(s), clean=0, didChange=#{changed_diags.size} unsaved diagnostic(s), API workspace/symbol+hover+definition passed, disk=unchanged." EXIT_OK else EXIT_FAILED diff --git a/spec/amber_lsp/controller_spec.cr b/spec/amber_lsp/controller_spec.cr index 41c497d..3cca372 100644 --- a/spec/amber_lsp/controller_spec.cr +++ b/spec/amber_lsp/controller_spec.cr @@ -32,6 +32,9 @@ describe AmberLSP::Controller do text_doc_sync["openClose"].as_bool.should be_true text_doc_sync["change"].as_i.should eq(1) text_doc_sync["save"]["includeText"].as_bool.should be_true + capabilities["workspaceSymbolProvider"].as_bool.should be_true + capabilities["hoverProvider"].as_bool.should be_true + capabilities["definitionProvider"].as_bool.should be_true # Check serverInfo server_info = result["serverInfo"] @@ -146,7 +149,7 @@ describe AmberLSP::Controller do request = { "jsonrpc" => "2.0", "id" => 99, - "method" => "textDocument/hover", + "method" => "textDocument/references", }.to_json response = server.controller.handle(request, server) diff --git a/spec/amber_lsp/lookup/api_index_service_spec.cr b/spec/amber_lsp/lookup/api_index_service_spec.cr new file mode 100644 index 0000000..055502f --- /dev/null +++ b/spec/amber_lsp/lookup/api_index_service_spec.cr @@ -0,0 +1,56 @@ +require "../spec_helper" + +require "../../../src/amber_lsp/lookup/api_index_service" +require "../../../src/amber_lsp/lookup/normalize_crystal_docs" + +describe AmberLSP::Lookup::APIIndexService do + it "keeps an invalidated background build from replacing a newer index" do + with_tempdir do |project| + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_small.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + project, + "project", + "fixture_project", + "fixture-key", + [] of String, + ).perform + stale_layer = AmberLSP::Lookup::CachedAPIIndexLayer.new(layer, "stale") + fresh_layer = AmberLSP::Lookup::CachedAPIIndexLayer.new(layer, "fresh") + build_started = Channel(Int32).new(2) + release_stale_build = Channel(Nil).new(1) + release_fresh_build = Channel(Nil).new(1) + build_count = 0 + layer_builder = ->(_root : String) { + build_count += 1 + current_build = build_count + build_started.send(current_build) + if current_build == 1 + release_stale_build.receive + [stale_layer] + else + release_fresh_build.receive + [fresh_layer] + end + } + service = AmberLSP::Lookup::APIIndexService.new(File.join(project, ".cache"), layer_builder) + + service.layers_for(project).should be_nil + build_started.receive.should eq(1) + service.invalidate(project) + service.layers_for(project).should be_nil + build_started.receive.should eq(2) + + release_fresh_build.send(nil) + 100.times do + break if service.layers_for(project) + Fiber.yield + end + service.layers_for(project).try(&.first.freshness).should eq("fresh") + + release_stale_build.send(nil) + Fiber.yield + service.layers_for(project).try(&.first.freshness).should eq("fresh") + end + end +end diff --git a/spec/amber_lsp/lookup/index_cache_spec.cr b/spec/amber_lsp/lookup/index_cache_spec.cr index 5c22069..b23f2be 100644 --- a/spec/amber_lsp/lookup/index_cache_spec.cr +++ b/spec/amber_lsp/lookup/index_cache_spec.cr @@ -91,4 +91,24 @@ describe "AmberLSP::Lookup::APIIndexCache#write_layer and #load_layer" do Dir.glob(File.join(cache_root, "**", "*.tmp-*")).should be_empty end end + + it "caches unavailable build results per project for warm retries" do + with_tempdir do |root| + cache_root = File.join(root, "cache") + first_project = File.join(root, "first_project") + second_project = File.join(root, "second_project") + first_cache = AmberLSP::Lookup::APIIndexCache.new(cache_root, first_project) + second_cache = AmberLSP::Lookup::APIIndexCache.new(cache_root, second_project) + + first_cache.write_failure("library", "gemma", "gemma-key", "missing dependency") + + cached_failure = first_cache.load_layer("library", "gemma", "gemma-key") + other_project = second_cache.load_layer("library", "gemma", "gemma-key") + + cached_failure.freshness.should eq("unavailable") + cached_failure.failure_reason.should eq("missing dependency") + other_project.freshness.should eq("unavailable") + other_project.failure_reason.should be_nil + end + end end diff --git a/spec/amber_lsp/lookup/lookup_command_spec.cr b/spec/amber_lsp/lookup/lookup_command_spec.cr index 89d090a..98600e4 100644 --- a/spec/amber_lsp/lookup/lookup_command_spec.cr +++ b/spec/amber_lsp/lookup/lookup_command_spec.cr @@ -19,6 +19,22 @@ describe AmberLSP::Lookup::ExtractLookupQueryAtPosition do query.not_nil!.receiver.should eq("user") end + it "extracts the receiver suffix when the line has preceding text" do + source = "# Hover target: CLIProject::User#id\n" + query = AmberLSP::Lookup::ExtractLookupQueryAtPosition.new(source, 0, 33).perform + + query.should_not be_nil + query.not_nil!.query.should eq("CLIProject::User#id") + end + + it "uses UTF-16 character offsets when the line has non-ASCII text" do + source = "😀😀😀😀 FixtureAPI::User#id\n" + query = AmberLSP::Lookup::ExtractLookupQueryAtPosition.new(source, 0, 28).perform + + query.should_not be_nil + query.not_nil!.query.should eq("FixtureAPI::User#id") + end + it "rejects positions without an identifier" do query = AmberLSP::Lookup::ExtractLookupQueryAtPosition.new("User.\n", 0, 5).perform diff --git a/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr b/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr new file mode 100644 index 0000000..ebda936 --- /dev/null +++ b/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr @@ -0,0 +1,140 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/normalize_crystal_docs" + +describe AmberLSP::Controller do + it "serves workspace symbols, hover, and definitions through the controller" do + with_lsp_lookup_project do |project, service| + controller = AmberLSP::Controller.new(service) + server = AmberLSP::Server.new(IO::Memory.new, IO::Memory.new) + initialize_lookup_controller(controller, server, project) + uri = "file://#{File.join(project, "src", "models", "user.cr")}" + + workspace_response = controller.handle( + lsp_lookup_request(1, "workspace/symbol", {"query" => "FixtureAPI::User#id"}), + server, + ) + workspace_json = JSON.parse(workspace_response.not_nil!) + + hover_response = controller.handle( + lsp_lookup_request(2, "textDocument/hover", { + "textDocument" => {"uri" => uri}, + "position" => {"line" => 0, "character" => 18}, + }), + server, + ) + hover_json = JSON.parse(hover_response.not_nil!) + + definition_response = controller.handle( + lsp_lookup_request(3, "textDocument/definition", { + "textDocument" => {"uri" => uri}, + "position" => {"line" => 0, "character" => 18}, + }), + server, + ) + definition_json = JSON.parse(definition_response.not_nil!) + + workspace_json["result"].as_a.size.should eq(1) + workspace_symbol = workspace_json["result"].as_a.first + workspace_symbol["name"].as_s.should eq("FixtureAPI::User#id()") + workspace_symbol["data"]["freshness"].as_s.should eq("fresh") + + hover_value = hover_json["result"]["contents"]["value"].as_s + hover_value.should contain("FixtureAPI::User#id() : Int64 | Nil") + hover_value.should contain("The user's primary key.") + hover_value.should contain("Layer: fixture_project") + hover_value.should contain("Freshness: fresh") + hover_value.should contain("The ID getter is generated from the persisted table column.") + + definition_json["result"].as_a.first["uri"].as_s.should eq("file://#{File.join(project, "src", "models", "user.cr")}") + definition_json["result"].as_a.first["range"]["start"]["line"].as_i.should eq(11) + end + end + + it "returns unavailable hover data while the index builds in a background fiber" do + with_lsp_lookup_project do |project, _service| + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_small.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + project, + "project", + "fixture_project", + "fixture-key", + [] of String, + ).perform + cached_layer = AmberLSP::Lookup::CachedAPIIndexLayer.new(layer, "fresh") + build_started = Channel(Nil).new(1) + release_build = Channel(Nil).new(1) + layer_builder = ->(_root : String) { + build_started.send(nil) + release_build.receive + [cached_layer] + } + service = AmberLSP::Lookup::APIIndexService.new( + File.join(project, ".cache"), + layer_builder, + ) + controller = AmberLSP::Controller.new(service) + server = AmberLSP::Server.new(IO::Memory.new, IO::Memory.new) + initialize_lookup_controller(controller, server, project) + uri = "file://#{File.join(project, "src", "models", "user.cr")}" + + response = controller.handle( + lsp_lookup_request(1, "textDocument/hover", { + "textDocument" => {"uri" => uri}, + "position" => {"line" => 0, "character" => 18}, + }), + server, + ) + hover_value = JSON.parse(response.not_nil!)["result"]["contents"]["value"].as_s + hover_value.should contain("API index unavailable") + hover_value.should contain("Freshness: unavailable") + + build_started.receive + release_build.send(nil) + 100.times do + break if service.layers_for(project) + Fiber.yield + end + service.layers_for(project).should_not be_nil + end + end +end + +private def initialize_lookup_controller( + controller : AmberLSP::Controller, + server : AmberLSP::Server, + project : String, +) : Nil + request = lsp_lookup_request(0, "initialize", {"rootUri" => "file://#{project}"}) + controller.handle(request, server).should_not be_nil +end + +private def lsp_lookup_request(id : Int32, method : String, params) : String + {"jsonrpc" => "2.0", "id" => id, "method" => method, "params" => params}.to_json +end + +private def with_lsp_lookup_project(&) + with_tempdir do |project| + source_root = File.join(project, "src", "models") + api_card_root = File.join(project, ".amber-lsp", "api") + Dir.mkdir_p(source_root) + Dir.mkdir_p(api_card_root) + File.write(File.join(project, "shard.yml"), "name: fixture_project\nversion: 1.5.0\n") + File.write(File.join(source_root, "user.cr"), "FixtureAPI::User#id\n") + card_fixture = File.join(Dir.current, "spec", "fixtures", "api_lookup", "cards", "project", ".amber-lsp") + FileUtils.cp_r(card_fixture, project) + + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_small.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + project, + "project", + "fixture_project", + "fixture-key", + [] of String, + ).perform + service = AmberLSP::Lookup::APIIndexService.new(File.join(project, ".cache")) + service.seed(project, [AmberLSP::Lookup::CachedAPIIndexLayer.new(layer, "fresh")]) + yield project, service + end +end diff --git a/src/amber_lsp/controller.cr b/src/amber_lsp/controller.cr index e2a7062..189d053 100644 --- a/src/amber_lsp/controller.cr +++ b/src/amber_lsp/controller.cr @@ -2,6 +2,13 @@ require "json" require "set" require "uri" +require "./lookup/api_cards" +require "./lookup/api_index_service" +require "./lookup/lsp_models" +require "./lookup/resolve_api_query" +require "./lookup/answer_api_query" +require "./lookup/extract_lookup_query_at_position" + module AmberLSP struct IncomingTextDocument include JSON::Serializable @@ -16,6 +23,13 @@ module AmberLSP getter text : String end + struct IncomingPosition + include JSON::Serializable + + getter line : Int32 = 0 + getter character : Int32 = 0 + end + struct IncomingParams include JSON::Serializable @@ -26,7 +40,9 @@ module AmberLSP @[JSON::Field(key: "textDocument")] getter text_document : IncomingTextDocument? = nil @[JSON::Field(key: "contentChanges")] - getter content_changes : Array(IncomingContentChange)? = nil + getter list_of_content_changes : Array(IncomingContentChange)? = nil + getter position : IncomingPosition? = nil + getter query : String? = nil getter text : String? = nil end @@ -43,7 +59,7 @@ module AmberLSP include JSON::Serializable @[JSON::Field(key: "includeText")] - getter include_text : Bool = true + getter should_include_text_on_save : Bool = true def initialize end @@ -54,7 +70,7 @@ module AmberLSP include JSON::Serializable @[JSON::Field(key: "openClose")] - getter open_close : Bool = true + getter can_open_and_close_documents : Bool = true getter change : Int32 = 1 getter save : SaveOptions @@ -68,8 +84,19 @@ module AmberLSP @[JSON::Field(key: "textDocumentSync")] getter text_document_sync : TextDocumentSyncOptions - - def initialize(@text_document_sync : TextDocumentSyncOptions = TextDocumentSyncOptions.new) + @[JSON::Field(key: "workspaceSymbolProvider")] + getter can_provide_workspace_symbols : Bool = true + @[JSON::Field(key: "hoverProvider")] + getter can_provide_hover : Bool = true + @[JSON::Field(key: "definitionProvider")] + getter can_provide_definitions : Bool = true + + def initialize( + @text_document_sync : TextDocumentSyncOptions = TextDocumentSyncOptions.new, + @can_provide_workspace_symbols : Bool = true, + @can_provide_hover : Bool = true, + @can_provide_definitions : Bool = true, + ) end end @@ -77,7 +104,8 @@ module AmberLSP struct ServerInfo include JSON::Serializable - getter name : String = "amber-lsp" + @[JSON::Field(key: "name")] + getter server_name : String = "amber-lsp" getter version : String = AmberLSP::VERSION def initialize @@ -152,9 +180,10 @@ module AmberLSP include JSON::Serializable getter uri : String - getter diagnostics : Array(Rules::LSPDiagnostic) + @[JSON::Field(key: "diagnostics")] + getter list_of_diagnostics : Array(Rules::LSPDiagnostic) - def initialize(@uri : String, @diagnostics : Array(Rules::LSPDiagnostic)) + def initialize(@uri : String, @list_of_diagnostics : Array(Rules::LSPDiagnostic)) end end @@ -198,8 +227,9 @@ module AmberLSP @analyzer : Analyzer? = nil @last_coverage_status : String? = nil @uris_with_current_diagnostics = Set(String).new + @roots_with_lookup_pending_message = Set(String).new - def initialize + def initialize(@lookup_index_service : Lookup::APIIndexService = Lookup::APIIndexService.new) @document_store = DocumentStore.new end @@ -225,6 +255,12 @@ module AmberLSP when "textDocument/didClose" handle_did_close(message.params, server) nil + when "workspace/symbol" + handle_workspace_symbol(id, message.params, server) + when "textDocument/hover" + handle_hover(id, message.params, server) + when "textDocument/definition" + handle_definition(id, message.params, server) when "shutdown" handle_shutdown(id) when "exit" @@ -262,6 +298,210 @@ module AmberLSP @project_root_path = project_root_path end + private def handle_workspace_symbol( + id : Int64 | String | Nil, + params : IncomingParams?, + server : Server, + ) : String? + return nil unless id + + query = params.try(&.query) || "" + answer, card_collection, index_is_ready = answer_for_lookup(query) + log_lookup_index_pending(server) unless index_is_ready + list_of_symbols = [] of Lookup::LSPWorkspaceSymbol + + answer.list_of_entries.each do |entry| + location = lsp_location(entry.source_path, entry.source_line) + notes = card_collection.matching_notes(query) + symbol_data = Lookup::LSPWorkspaceSymbolData.new( + answer.status, + answer.freshness, + entry.source_layer, + notes, + ) + list_of_symbols << Lookup::LSPWorkspaceSymbol.new( + "#{entry.owner}##{entry.name}#{entry.args_string}", + 6, + location, + entry.owner, + symbol_data, + ) + end + + if type_summary = answer.type_summary + kind = case type_summary.kind + when "class" then 5 + when "module" then 2 + when "struct" then 23 + else 1 + end + location = lsp_location(type_summary.location_path, type_summary.location_line) + symbol_data = Lookup::LSPWorkspaceSymbolData.new( + answer.status, + answer.freshness, + type_summary.source_layer, + card_collection.matching_notes(query), + ) + list_of_symbols << Lookup::LSPWorkspaceSymbol.new( + type_summary.name, + kind, + location, + nil, + symbol_data, + ) + end + + Lookup::LSPWorkspaceSymbolResponse.new(id, list_of_symbols).to_json + end + + private def handle_hover( + id : Int64 | String | Nil, + params : IncomingParams?, + server : Server, + ) : String? + return nil unless id + + extracted_query = query_at_position(params) + return Lookup::LSPHoverResponse.new(id, nil).to_json unless extracted_query + + query = extracted_query[0] + answer, card_collection, index_is_ready = answer_for_lookup(query) + log_lookup_index_pending(server) unless index_is_ready + markdown = hover_markdown(query, answer, card_collection) + hover = Lookup::DescribeAPIForHover.new(Lookup::LSPMarkupContent.new(markdown)) + Lookup::LSPHoverResponse.new(id, hover).to_json + end + + private def handle_definition( + id : Int64 | String | Nil, + params : IncomingParams?, + server : Server, + ) : String? + return nil unless id + + extracted_query = query_at_position(params) + return Lookup::LSPDefinitionResponse.new(id, nil).to_json unless extracted_query + + answer, _card_collection, index_is_ready = answer_for_lookup(extracted_query[0]) + unless index_is_ready + log_lookup_index_pending(server) + return Lookup::LSPDefinitionResponse.new(id, nil).to_json + end + + list_of_locations = answer.list_of_entries.map do |entry| + lsp_location(entry.source_path, entry.source_line) + end + if list_of_locations.empty? + if type_summary = answer.type_summary + list_of_locations << lsp_location(type_summary.location_path, type_summary.location_line) + end + end + + locations = list_of_locations.empty? ? nil : list_of_locations + Lookup::LSPDefinitionResponse.new(id, locations).to_json + end + + private def query_at_position(params : IncomingParams?) : Tuple(String, String)? + return nil unless params + text_document = params.text_document + position = params.position + return nil unless text_document && position + uri = text_document.uri + return nil unless uri + + content = @document_store.get(uri) || File.read(uri_to_path(uri)) + extracted = Lookup::ExtractLookupQueryAtPosition.new(content, position.line, position.character).perform + extracted.try { |result| {result.query, uri} } + rescue ex : IO::Error + nil + end + + private def answer_for_lookup(query : String) : Tuple(Lookup::LookupAnswer, Lookup::APICardCollection, Bool) + project_root = @project_root_path || Dir.current + cached_layers = @lookup_index_service.layers_for(project_root) + card_collection = Lookup::LoadAPICards.new(project_root).perform + + if cached_layers + @roots_with_lookup_pending_message.delete(project_root) + list_of_index_layers = cached_layers.compact_map(&.layer) + resolution = Lookup::ResolveAPIQuery.new(query, list_of_index_layers).perform + answer = Lookup::AnswerAPIQuery.new(query, resolution, cached_layers).perform + return {answer, card_collection, true} + end + + layer_state = Lookup::APIIndexLayerState.new( + "project", + File.basename(project_root), + "", + "unavailable", + "API index is being built in the background", + ) + answer = Lookup::LookupAnswer.new( + query, + "unknown", + "unavailable", + nil, + [] of Lookup::APIIndexMethod, + [layer_state], + ) + {answer, card_collection, false} + end + + private def hover_markdown( + query : String, + answer : Lookup::LookupAnswer, + card_collection : Lookup::APICardCollection, + ) : String + lines = [] of String + answer.list_of_entries.each do |entry| + argument_string = entry.args_string.starts_with?('(') ? entry.args_string : "(#{entry.args_string})" + return_type = entry.return_type.gsub("::Nil", "Nil") + lines << "```crystal" + lines << "#{entry.owner}##{entry.name}#{argument_string} : #{return_type}" + lines << "```" + if doc_line = entry.doc_line + lines << doc_line + end + lines << "Layer: #{entry.source_layer}" + end + + if answer.list_of_entries.empty? + if type_summary = answer.type_summary + lines << "```crystal" + lines << "#{type_summary.kind} #{type_summary.name}" + lines << "```" + lines << "Layer: #{type_summary.source_layer}" + else + lines << "```crystal" + lines << (answer.freshness == "unavailable" ? "API index unavailable" : "No indexed API found for #{query}") + lines << "```" + end + end + + lines << "Status: #{answer.status}" + lines << "Freshness: #{answer.freshness}" + card_collection.matching_notes(query).each do |note| + lines << "Note (#{note.symbol}): #{note.text}" + end + lines.join("\n\n") + end + + private def lsp_location(path : String, line_number : Int32) : Lookup::LSPLocation + start_position = Lookup::LSPPosition.new(Math.max(line_number - 1, 0), 0) + end_position = Lookup::LSPPosition.new(Math.max(line_number - 1, 0), 0) + range = Lookup::LSPRange.new(start_position, end_position) + Lookup::LSPLocation.new("file://#{URI.encode_path(path)}", range) + end + + private def log_lookup_index_pending(server : Server) : Nil + project_root = @project_root_path || Dir.current + return if @roots_with_lookup_pending_message.includes?(project_root) + + @roots_with_lookup_pending_message.add(project_root) + params = LogMessageParams.new(3, "amber-lsp: API index unavailable while a background build is running") + server.write_notification(LogMessageNotification.new(params).to_json) + end + private def handle_did_open(params : IncomingParams?, server : Server) : Nil return unless params @@ -279,7 +519,7 @@ module AmberLSP private def handle_did_change(params : IncomingParams?, server : Server) : Nil return unless params uri = params.text_document.try(&.uri) - text = params.content_changes.try(&.last?).try(&.text) + text = params.list_of_content_changes.try(&.last?).try(&.text) return unless uri && text @document_store.update(uri, text) @@ -295,6 +535,10 @@ module AmberLSP uri = text_document.uri return unless uri + if project_root = @project_root_path + @lookup_index_service.invalidate(project_root) + end + text = params.text if text @document_store.update(uri, text) diff --git a/src/amber_lsp/lookup/api_index_service.cr b/src/amber_lsp/lookup/api_index_service.cr new file mode 100644 index 0000000..5a6f894 --- /dev/null +++ b/src/amber_lsp/lookup/api_index_service.cr @@ -0,0 +1,87 @@ +require "mutex" +require "./build_layered_api_index" + +module AmberLSP::Lookup + class APIIndexService + def initialize( + @cache_root : String = APIIndexCache.default_root, + @layer_builder : Proc(String, Array(CachedAPIIndexLayer))? = nil, + ) + @mutex = Mutex.new + @layers_by_root = {} of String => Array(CachedAPIIndexLayer) + @building_generation_by_root = {} of String => Int64 + @latest_generation_by_root = {} of String => Int64 + end + + def layers_for(root_path : String) : Array(CachedAPIIndexLayer)? + normalized_root = File.expand_path(root_path) + cached_layers : Array(CachedAPIIndexLayer)? = nil + build_generation : Int64? = nil + + @mutex.synchronize do + cached_layers = @layers_by_root[normalized_root]? + unless cached_layers || @building_generation_by_root.has_key?(normalized_root) + next_generation = (@latest_generation_by_root[normalized_root]? || 0_i64) + 1 + @latest_generation_by_root[normalized_root] = next_generation + @building_generation_by_root[normalized_root] = next_generation + build_generation = next_generation + end + end + + if generation = build_generation + spawn do + build_and_store(normalized_root, generation) + end + end + + cached_layers + end + + def seed(root_path : String, layers : Array(CachedAPIIndexLayer)) : Nil + normalized_root = File.expand_path(root_path) + @mutex.synchronize do + @layers_by_root[normalized_root] = layers + @latest_generation_by_root[normalized_root] = (@latest_generation_by_root[normalized_root]? || 0_i64) + 1 + @building_generation_by_root.delete(normalized_root) + end + end + + def invalidate(root_path : String) : Nil + normalized_root = File.expand_path(root_path) + @mutex.synchronize do + @layers_by_root.delete(normalized_root) + @latest_generation_by_root[normalized_root] = (@latest_generation_by_root[normalized_root]? || 0_i64) + 1 + @building_generation_by_root.delete(normalized_root) + end + end + + private def build_and_store(root_path : String, generation : Int64) : Nil + list_of_layers = if builder = @layer_builder + builder.call(root_path) + else + BuildLayeredAPIIndex.new(root_path, @cache_root).perform + end + @mutex.synchronize do + if @building_generation_by_root[root_path]? == generation + @layers_by_root[root_path] = list_of_layers + @building_generation_by_root.delete(root_path) + end + end + rescue ex : Exception + unavailable_layer = CachedAPIIndexLayer.new( + nil, + "unavailable", + ex.message || "API index build failed", + "project", + File.basename(root_path), + "", + ) + @mutex.synchronize do + if @building_generation_by_root[root_path]? == generation + @layers_by_root[root_path] = [unavailable_layer] + @building_generation_by_root.delete(root_path) + end + end + end + end +end diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr index abd1633..4a016a0 100644 --- a/src/amber_lsp/lookup/build_layered_api_index.cr +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -220,7 +220,7 @@ module AmberLSP::Lookup environment_overrides : Hash(String, String), ) : CachedAPIIndexLayer cached_layer = cache.load_layer(layer_kind, layer_name, layer_key) - return cached_layer if cached_layer.freshness == "fresh" + return cached_layer if cached_layer.freshness == "fresh" || cached_layer.failure_reason begin docs_json = BuildCrystalDocsJSON.new( @@ -243,9 +243,13 @@ module AmberLSP::Lookup cache.write_layer(layer) CachedAPIIndexLayer.new(layer, "fresh") rescue ex : APIIndexBuildError | JSON::ParseException - return cached_layer if cached_layer.layer + failure_reason = ex.message || "crystal-alpha docs could not build this API layer" + cache.write_failure(layer_kind, layer_name, layer_key, failure_reason) + if stale_layer = cached_layer.layer + return CachedAPIIndexLayer.new(stale_layer, "stale", failure_reason, layer_kind, layer_name, layer_key) + end - CachedAPIIndexLayer.new(nil, "unavailable", ex.message, layer_kind, layer_name, layer_key) + CachedAPIIndexLayer.new(nil, "unavailable", failure_reason, layer_kind, layer_name, layer_key) end end diff --git a/src/amber_lsp/lookup/extract_lookup_query_at_position.cr b/src/amber_lsp/lookup/extract_lookup_query_at_position.cr new file mode 100644 index 0000000..b662336 --- /dev/null +++ b/src/amber_lsp/lookup/extract_lookup_query_at_position.cr @@ -0,0 +1,63 @@ +module AmberLSP::Lookup + struct LookupAtQuery + getter query : String + getter token : String + getter receiver : String? + + def initialize(@query : String, @token : String, @receiver : String?) + end + end + + class ExtractLookupQueryAtPosition + def initialize(@source : String, @line_number : Int32, @column_number : Int32) + end + + def perform : LookupAtQuery? + return nil if @line_number < 0 || @column_number < 0 + + line = @source.lines[@line_number]?.try(&.rstrip("\r\n")) + return nil unless line + + column = byte_offset_for_utf16_column(line, @column_number) + before_cursor = line.byte_slice(0, column) + after_cursor = line.byte_slice(column, line.bytesize - column) + left_token = before_cursor.match(/[A-Za-z0-9_!?]+\z/).try(&.[0]) || "" + right_token = after_cursor.match(/\A[A-Za-z0-9_!?]+/).try(&.[0]) || "" + token = left_token + right_token + return nil if token.empty? + + receiver_prefix = before_cursor.byte_slice(0, before_cursor.bytesize - left_token.bytesize) + receiver_match = receiver_prefix.match(/([A-Za-z0-9_:@.!?()\[\]]+)([.#])\s*\z/) + return LookupAtQuery.new(token, token, nil) unless receiver_match + + receiver = receiver_match[1].strip + separator = receiver_match[2] + if constant_receiver?(receiver) + LookupAtQuery.new("#{receiver}#{separator}#{token}", token, receiver) + else + LookupAtQuery.new(token, token, receiver) + end + end + + private def constant_receiver?(receiver : String) : Bool + receiver.matches?(/\A(?:::)?[A-Z][A-Za-z0-9_]*(?:::[A-Z][A-Za-z0-9_]*)*(?:\([A-Za-z0-9_:, ?|&]+\))?\z/) + end + + private def byte_offset_for_utf16_column(line : String, column_number : Int32) : Int32 + current_utf16_column = 0 + byte_offset = 0 + + line.each_char do |character| + break if current_utf16_column >= column_number + + character_utf16_width = character.ord > 0xFFFF_u32 ? 2 : 1 + break if current_utf16_column + character_utf16_width > column_number + + current_utf16_column += character_utf16_width + byte_offset += character.to_s.bytesize + end + + byte_offset + end + end +end diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr index b094426..d4fce40 100644 --- a/src/amber_lsp/lookup/index_cache.cr +++ b/src/amber_lsp/lookup/index_cache.cr @@ -110,6 +110,26 @@ module AmberLSP::Lookup end end + # :nodoc: + struct APIIndexCacheFailure + include JSON::Serializable + + getter layer_kind : String + getter layer_name : String + getter layer_key : String + getter failure_reason : String + getter created_at_unix : Int64 + + def initialize( + @layer_kind : String, + @layer_name : String, + @layer_key : String, + @failure_reason : String, + @created_at_unix : Int64, + ) + end + end + class APIIndexCache def initialize(@cache_root : String, @project_root_path : String) end @@ -125,6 +145,9 @@ module AmberLSP::Lookup layer_path = File.join(directory_path, "#{layer.layer_key}.json") write_atomically(layer_path, layer.to_json) + failure_path = failure_marker_path(layer.layer_kind, layer.layer_name, layer.layer_key) + File.delete(failure_path) if File.file?(failure_path) + latest_path = File.join(directory_path, "latest.json") write_atomically(latest_path, APIIndexCachePointer.new(layer.layer_key).to_json) layer_path @@ -140,6 +163,12 @@ module AmberLSP::Lookup end end + if failure = load_cached_failure(layer_kind, layer_name, expected_key) + stale_layer = latest_layer(directory_path, layer_kind, layer_name) + freshness = stale_layer ? "stale" : "unavailable" + return CachedAPIIndexLayer.new(stale_layer, freshness, failure.failure_reason, layer_kind, layer_name, expected_key) + end + latest_path = File.join(directory_path, "latest.json") return CachedAPIIndexLayer.new(nil, "unavailable", nil, layer_kind, layer_name, expected_key) unless File.file?(latest_path) @@ -153,6 +182,56 @@ module AmberLSP::Lookup CachedAPIIndexLayer.new(nil, "unavailable", ex.message, layer_kind, layer_name, expected_key) end + def write_failure( + layer_kind : String, + layer_name : String, + layer_key : String, + failure_reason : String, + ) : Nil + marker = APIIndexCacheFailure.new( + layer_kind, + layer_name, + layer_key, + failure_reason, + Time.utc.to_unix, + ) + write_atomically(failure_marker_path(layer_kind, layer_name, layer_key), marker.to_json) + end + + private def load_cached_failure( + layer_kind : String, + layer_name : String, + layer_key : String, + ) : APIIndexCacheFailure? + marker_path = failure_marker_path(layer_kind, layer_name, layer_key) + return nil unless File.file?(marker_path) + + marker = APIIndexCacheFailure.from_json(File.read(marker_path)) + return nil unless marker.layer_kind == layer_kind + return nil unless marker.layer_name == layer_name + return nil unless marker.layer_key == layer_key + return nil if Time.utc.to_unix - marker.created_at_unix > 300 + + marker + rescue ex : JSON::ParseException | IO::Error + nil + end + + private def latest_layer(directory_path : String, layer_kind : String, layer_name : String) : APIIndexLayer? + latest_path = File.join(directory_path, "latest.json") + return nil unless File.file?(latest_path) + + pointer = APIIndexCachePointer.from_json(File.read(latest_path)) + load_matching_layer( + File.join(directory_path, "#{pointer.key}.json"), + layer_kind, + layer_name, + pointer.key, + ) + rescue ex : JSON::ParseException | IO::Error + nil + end + private def load_matching_layer( layer_path : String, expected_kind : String, @@ -183,11 +262,24 @@ module AmberLSP::Lookup end end + private def failure_marker_path(layer_kind : String, layer_name : String, layer_key : String) : String + project_key = Digest::SHA256.hexdigest(File.expand_path(@project_root_path)) + File.join( + @cache_root, + "failures", + project_key, + safe_path_component(layer_kind), + safe_path_component(layer_name), + "#{layer_key}.json", + ) + end + private def safe_path_component(value : String) : String value.gsub(/[^A-Za-z0-9_-]/, "_") end private def write_atomically(path : String, contents : String) : Nil + Dir.mkdir_p(File.dirname(path)) temp_path = "#{path}.tmp-#{Random::Secure.hex(12)}" begin File.write(temp_path, contents) diff --git a/src/amber_lsp/lookup/lookup_command.cr b/src/amber_lsp/lookup/lookup_command.cr index ce43e4a..c671fda 100644 --- a/src/amber_lsp/lookup/lookup_command.cr +++ b/src/amber_lsp/lookup/lookup_command.cr @@ -4,21 +4,22 @@ require "./answer_api_query" require "./build_layered_api_index" require "./resolve_api_query" require "./verify_api_query" +require "./extract_lookup_query_at_position" module AmberLSP::Lookup struct LookupCLIOptions getter query : String? getter at_location : String? getter root_path : String - getter json : Bool - getter verify : Bool + getter should_output_json : Bool + getter should_verify_api_query : Bool def initialize( @query : String?, @at_location : String?, @root_path : String, - @json : Bool, - @verify : Bool, + @should_output_json : Bool, + @should_verify_api_query : Bool, ) end end @@ -27,7 +28,8 @@ module AmberLSP::Lookup include JSON::Serializable getter query : String - getter status : String + @[JSON::Field(key: "status")] + getter lookup_status : String getter freshness : String @[JSON::Field(emit_null: true)] getter type_summary : APIIndexType? @@ -50,7 +52,7 @@ module AmberLSP::Lookup def initialize( @query : String, - @status : String, + @lookup_status : String, @freshness : String, @type_summary : APIIndexType?, @list_of_entries : Array(APIIndexMethod), @@ -65,51 +67,6 @@ module AmberLSP::Lookup end end - struct LookupAtQuery - getter query : String - getter token : String - getter receiver : String? - - def initialize(@query : String, @token : String, @receiver : String?) - end - end - - class ExtractLookupQueryAtPosition - def initialize(@source : String, @line_number : Int32, @column_number : Int32) - end - - def perform : LookupAtQuery? - return nil if @line_number < 0 || @column_number < 0 - - line = @source.lines[@line_number]?.try(&.rstrip("\r\n")) - return nil unless line - - column = Math.min(@column_number, line.bytesize) - before_cursor = line.byte_slice(0, column) - after_cursor = line.byte_slice(column, line.bytesize - column) - left_token = before_cursor.match(/[A-Za-z0-9_!?]+\z/).try(&.[0]) || "" - right_token = after_cursor.match(/\A[A-Za-z0-9_!?]+/).try(&.[0]) || "" - token = left_token + right_token - return nil if token.empty? - - receiver_prefix = before_cursor.byte_slice(0, before_cursor.bytesize - left_token.bytesize) - receiver_match = receiver_prefix.match(/(.+)([.#])\s*\z/) - return LookupAtQuery.new(token, token, nil) unless receiver_match - - receiver = receiver_match[1].strip - separator = receiver_match[2] - if constant_receiver?(receiver) - LookupAtQuery.new("#{receiver}#{separator}#{token}", token, receiver) - else - LookupAtQuery.new(token, token, receiver) - end - end - - private def constant_receiver?(receiver : String) : Bool - receiver.matches?(/\A(?:::)?[A-Z][A-Za-z0-9_]*(?:::[A-Z][A-Za-z0-9_]*)*(?:\([A-Za-z0-9_:, ?|&]+\))?\z/) - end - end - class RunLookupCommand def initialize( @arguments : Array(String), @@ -133,12 +90,12 @@ module AmberLSP::Lookup list_of_index_layers = cached_layers.compact_map(&.layer) resolution = ResolveAPIQuery.new(query, list_of_index_layers).perform answer = AnswerAPIQuery.new(query, resolution, cached_layers).perform - probe = options.verify ? VerifyAPIQuery.new(options.root_path, query, resolution, @compiler_command).perform : nil + probe = options.should_verify_api_query ? VerifyAPIQuery.new(options.root_path, query, resolution, @compiler_command).perform : nil note_list = card_collection.matching_notes(query) hint_list = probe.try(&.output).try { |output| card_collection.matching_error_hints(output) } || [] of APICardErrorHint cli_answer = make_cli_answer(answer, probe, note_list, hint_list, card_collection.list_of_errors) - if options.json + if options.should_output_json @stdout.puts cli_answer.to_json else print_plain_answer(cli_answer, probe) @@ -149,15 +106,15 @@ module AmberLSP::Lookup return 2 end - exit_code_for(cli_answer.status) + exit_code_for(cli_answer.lookup_status) rescue ex : Exception @stderr.puts("amber-lsp lookup failed: #{ex.message || "unknown error"}") 2 end private def parse_options : LookupCLIOptions - query = nil.as(String?) - at_location = nil.as(String?) + query : String? = nil + at_location : String? = nil root_path = Dir.current json = false verify = false @@ -193,9 +150,12 @@ module AmberLSP::Lookup end private def lookup_query(options : LookupCLIOptions) : String - return options.query.not_nil! if options.query + if query = options.query + return query + end - at_location = options.at_location.not_nil! + at_location = options.at_location + raise ArgumentError.new("lookup requires a query or --at FILE:LINE:COL") unless at_location match = at_location.match(/\A(.+):(\d+):(\d+)\z/) raise ArgumentError.new("--at requires FILE:LINE:COL") unless match @@ -221,7 +181,7 @@ module AmberLSP::Lookup status = answer.status list_of_entries = answer.list_of_entries verification_status = probe.try(&.status) - failure_reason = nil.as(String?) + failure_reason : String? = nil if probe case probe.status @@ -256,7 +216,7 @@ module AmberLSP::Lookup end private def print_plain_answer(answer : LookupCLIAnswer, probe : APIProbeResult?) : Nil - @stdout.puts("amber-lsp lookup: #{answer.status} (#{answer.freshness})") + @stdout.puts("amber-lsp lookup: #{answer.lookup_status} (#{answer.freshness})") if type_summary = answer.type_summary @stdout.puts("#{type_summary.name} (#{type_summary.kind}) — #{type_summary.location_path}:#{type_summary.location_line} [#{type_summary.source_layer}]") end diff --git a/src/amber_lsp/lookup/lsp_models.cr b/src/amber_lsp/lookup/lsp_models.cr new file mode 100644 index 0000000..fb8df0b --- /dev/null +++ b/src/amber_lsp/lookup/lsp_models.cr @@ -0,0 +1,137 @@ +require "json" + +require "./api_cards" + +module AmberLSP::Lookup + struct LSPPosition + include JSON::Serializable + + getter line : Int32 + getter character : Int32 + + def initialize(@line : Int32 = 0, @character : Int32 = 0) + end + end + + struct LSPRange + include JSON::Serializable + + getter start : LSPPosition + getter end : LSPPosition + + def initialize(@start : LSPPosition, @end : LSPPosition) + end + end + + struct LSPLocation + include JSON::Serializable + + getter uri : String + getter range : LSPRange + + def initialize(@uri : String, @range : LSPRange) + end + end + + struct LSPWorkspaceSymbolData + include JSON::Serializable + + @[JSON::Field(key: "status")] + getter lookup_status : String + getter freshness : String + getter source_layer : String + @[JSON::Field(key: "notes")] + getter list_of_api_card_notes : Array(APICardNote) + + def initialize( + @lookup_status : String, + @freshness : String, + @source_layer : String, + @list_of_api_card_notes : Array(APICardNote) = [] of APICardNote, + ) + end + end + + struct LSPWorkspaceSymbol + include JSON::Serializable + + @[JSON::Field(key: "name")] + getter symbol_name : String + @[JSON::Field(key: "kind")] + getter symbol_kind : Int32 + getter location : LSPLocation + @[JSON::Field(key: "containerName", emit_null: true)] + getter container_name : String? + @[JSON::Field(key: "data")] + getter lookup_data : LSPWorkspaceSymbolData + + def initialize( + @symbol_name : String, + @symbol_kind : Int32, + @location : LSPLocation, + @container_name : String?, + @lookup_data : LSPWorkspaceSymbolData, + ) + end + end + + struct LSPWorkspaceSymbolResponse + include JSON::Serializable + + getter jsonrpc : String = "2.0" + @[JSON::Field(emit_null: true)] + getter id : Int64 | String | Nil + @[JSON::Field(key: "result")] + getter list_of_workspace_symbols : Array(LSPWorkspaceSymbol) + + def initialize(@id : Int64 | String | Nil, @list_of_workspace_symbols : Array(LSPWorkspaceSymbol)) + end + end + + struct LSPMarkupContent + include JSON::Serializable + + @[JSON::Field(key: "kind")] + getter markup_kind : String = "markdown" + @[JSON::Field(key: "value")] + getter markdown_value : String + + def initialize(@markdown_value : String) + end + end + + struct DescribeAPIForHover + include JSON::Serializable + + getter contents : LSPMarkupContent + + def initialize(@contents : LSPMarkupContent) + end + end + + struct LSPHoverResponse + include JSON::Serializable + + getter jsonrpc : String = "2.0" + @[JSON::Field(emit_null: true)] + getter id : Int64 | String | Nil + @[JSON::Field(emit_null: true)] + getter result : DescribeAPIForHover? + + def initialize(@id : Int64 | String | Nil, @result : DescribeAPIForHover?) + end + end + + struct LSPDefinitionResponse + include JSON::Serializable + + getter jsonrpc : String = "2.0" + @[JSON::Field(emit_null: true)] + getter id : Int64 | String | Nil + @[JSON::Field(key: "result", emit_null: true)] + getter list_of_locations : Array(LSPLocation)? + + def initialize(@id : Int64 | String | Nil, @list_of_locations : Array(LSPLocation)?) + end + end +end From 8f706427126b33c4aa0eb44a6048cba79d1e982c Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 20:51:22 -0400 Subject: [PATCH 26/69] feat(amber-cli): add cached agent readiness check --- spec/commands/agent_hook_spec.cr | 341 +++++++++++++ .../templates/agent/amber-agent-hook | 471 +++++++++++++++++- 2 files changed, 800 insertions(+), 12 deletions(-) create mode 100644 spec/commands/agent_hook_spec.cr diff --git a/spec/commands/agent_hook_spec.cr b/spec/commands/agent_hook_spec.cr new file mode 100644 index 0000000..9c36bd4 --- /dev/null +++ b/spec/commands/agent_hook_spec.cr @@ -0,0 +1,341 @@ +require "../amber_cli_spec" +require "digest/sha256" +require "../../src/amber_cli/commands/setup_agent" + +record AgentHookFixture, project_root : String, hook_path : String, lsp_path : String, lsp_log : String, environment : Hash(String, String) + +module AgentHookSpecHelper + def self.create_ready_project(project_root : String, version : String = "1.0.0", coverage : String = "amber-lsp: covered no diagnostics", coverage_exit : String = "0") : AgentHookFixture + Dir.mkdir_p("src") + Dir.mkdir_p("bin") + Dir.mkdir_p(".amber/claude-marketplace/.claude-plugin") + Dir.mkdir_p(".amber/claude-marketplace/amber-lsp/.claude-plugin") + Dir.mkdir_p(".claude") + Dir.mkdir_p(".codex") + File.write("shard.yml", "name: hook_app\ntargets:\n app:\n main: src/app.cr\n") + File.write("src/app.cr", "puts :ready\n") + File.write(".amber/agent_setup.json", AmberCLI::Agent::AgentSetupManifest.new( + AmberCli::VERSION, + "1.0.0", + "3", + ).to_pretty_json + "\n") + File.write(".amber/claude-marketplace/.claude-plugin/marketplace.json", <<-JSON) + { + "name": "amber", + "owner": {"name": "Amber Framework"}, + "plugins": [{"name": "amber-lsp", "source": "./amber-lsp"}] + } + JSON + File.write(".amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json", <<-JSON) + { + "name": "amber-lsp", + "version": "1.0.0", + "description": "Language server for Amber V2 projects." + } + JSON + File.write(".amber/claude-marketplace/amber-lsp/.lsp.json", <<-JSON) + { + "amber": { + "command": "amber-lsp", + "extensionToLanguage": {".cr": "crystal"} + } + } + JSON + File.write(".claude/settings.json", <<-JSON) + { + "extraKnownMarketplaces": { + "amber": { + "source": {"source": "directory", "path": "./.amber/claude-marketplace"} + } + }, + "enabledPlugins": {"amber-lsp@amber": true} + } + JSON + + script = AmberCLI::Commands::SetupAgentCommand::AGENT_HOOK_SCRIPT.gsub("__AMBER_MAIN__", "src/app.cr") + hook_path = File.join(project_root, "bin/amber-agent-hook") + File.write(hook_path, script) + File.chmod(hook_path, 0o755) + + tools = File.join(project_root, "fake-tools") + Dir.mkdir_p(tools) + lsp_path = File.join(tools, "amber-lsp") + lsp_log = File.join(project_root, "amber-lsp.log") + write_lsp(lsp_path, version, coverage, coverage_exit) + environment = { + "PATH" => "#{tools}:/usr/bin:/bin", + "AMBER_LSP_BIN" => lsp_path, + "TEST_LSP_LOG" => lsp_log, + "TEST_VERSION" => version, + "TEST_COVERAGE" => coverage, + "TEST_COVERAGE_EXIT" => coverage_exit, + "TMPDIR" => File.join(project_root, "hook-cache"), + } + Dir.mkdir_p(environment["TMPDIR"]) + AgentHookFixture.new(project_root, hook_path, lsp_path, lsp_log, environment) + end + + def self.write_lsp(path : String, version : String, coverage : String, coverage_exit : String) : Nil + content = <<-SH + #!/bin/sh + printf '%s\\n' "$*" >> "$TEST_LSP_LOG" + case "$1" in + --version) printf 'amber-lsp %s\\n' "$TEST_VERSION"; exit 0 ;; + --check) printf '%s\\n' "$TEST_COVERAGE"; exit "$TEST_COVERAGE_EXIT" ;; + esac + exit 2 + SH + File.write(path, content) + File.chmod(path, 0o755) + write_sidecar(path) + end + + def self.write_sidecar(binary_path : String) : Nil + checksum = Digest::SHA256.hexdigest(File.read(binary_path)) + File.write("#{binary_path}.sha256", "#{checksum} #{binary_path}\n") + end + + def self.run_hook(fixture : AgentHookFixture, mode : String, payload : String = "{}", environment_overrides : Hash(String, String) = {} of String => String) : Tuple(Int32, String, String) + output = IO::Memory.new + errors = IO::Memory.new + environment = fixture.environment.merge(environment_overrides) + status = Process.run( + fixture.hook_path, + [mode], + input: IO::Memory.new(payload), + output: output, + error: errors, + env: environment, + ) + {status.exit_code, output.to_s, errors.to_s} + end +end + +describe "amber-agent-hook readiness and preflight" do + it "checks a verified binary and project coverage once, then serves the cached result under 300 ms" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + + first_check = AgentHookSpecHelper.run_hook(fixture, "check") + first_check[0].should eq(0) + first_check[1].should eq("Amber agent setup is ready.\n") + File.read(fixture.lsp_log).should contain("--version") + File.read(fixture.lsp_log).should contain("--check src/app.cr") + + previous_calls = File.read(fixture.lsp_log) + start_time = Time.instant + cached_check = AgentHookSpecHelper.run_hook(fixture, "check") + elapsed_milliseconds = (Time.instant - start_time).total_milliseconds + cached_check[0].should eq(0) + File.read(fixture.lsp_log).should eq(previous_calls) + elapsed_milliseconds.should be < 300.0 + end + end + + it "reports a missing binary in SessionStart and returns the same refusal from a Crystal edit" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + missing_binary = {"AMBER_LSP_BIN" => "/nonexistent"} + session = AgentHookSpecHelper.run_hook(fixture, "session", "{}", missing_binary) + preflight = AgentHookSpecHelper.run_hook( + fixture, + "pre", + {"tool_name" => "Edit", "tool_input" => {"file_path" => "/project/src/change.cr"}}.to_json, + missing_binary, + ) + + session[0].should eq(0) + session[1].should contain("The amber-lsp binary was not found at /nonexistent or on PATH.") + session[1].should contain("Do not edit Crystal files yet. Tell the user to run `amber setup:agent` in this project, then continue.") + preflight[0].should eq(2) + preflight[2].should eq(session[1]) + end + end + + it "uses AMBER_LSP_BIN as the selected executable and reports a single missing-binary item" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + + check = AgentHookSpecHelper.run_hook(fixture, "check", "{}", {"AMBER_LSP_BIN" => "/nonexistent"}) + + check[0].should eq(2) + check[1].lines.size.should eq(1) + check[1].should contain("The amber-lsp binary was not found at /nonexistent or on PATH.") + end + end + + it "finds amber-lsp on PATH when AMBER_LSP_BIN is unset" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + environment = fixture.environment.dup + environment.delete("AMBER_LSP_BIN") + status = Process.run(fixture.hook_path, ["check"], input: IO::Memory.new("{}"), output: IO::Memory.new, + error: IO::Memory.new, env: environment) + + status.exit_code.should eq(0) + end + end + + it "fails a release below the setup minimum before running project coverage" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root, version: "0.9.9") + + check = AgentHookSpecHelper.run_hook(fixture, "check") + + check[0].should eq(2) + check[1].should contain("The amber-lsp version 0.9.9 is below the required 1.0.0; update amber-lsp.") + File.read(fixture.lsp_log).should_not contain("--check") + end + end + + it "fails closed when the adjacent checksum is missing" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + File.delete("#{fixture.lsp_path}.sha256") + + check = AgentHookSpecHelper.run_hook(fixture, "check") + + check[0].should eq(2) + check[1].should contain("The amber-lsp checksum is missing or does not match") + File.read(fixture.lsp_log).should_not contain("--check") + end + end + + it "fails closed when the adjacent checksum does not match the binary" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + File.write("#{fixture.lsp_path}.sha256", "#{"0" * 64} amber-lsp\n") + + check = AgentHookSpecHelper.run_hook(fixture, "check") + + check[0].should eq(2) + check[1].should contain("The amber-lsp checksum is missing or does not match") + File.read(fixture.lsp_log).should_not contain("--check") + end + end + + it "reads the release checksum from prefix share when no adjacent checksum exists" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + File.delete("#{fixture.lsp_path}.sha256") + release_prefix = File.join(project_root, "release") + release_binary = File.join(release_prefix, "bin/amber-lsp") + Dir.mkdir_p(File.dirname(release_binary)) + File.copy(fixture.lsp_path, release_binary) + digest = Digest::SHA256.hexdigest(File.read(release_binary)) + checksum_dir = File.join(release_prefix, "share/amber_cli") + Dir.mkdir_p(checksum_dir) + File.write(File.join(checksum_dir, "checksums.txt"), "#{digest} amber-lsp\n") + + check = AgentHookSpecHelper.run_hook(fixture, "check", "{}", {"AMBER_LSP_BIN" => release_binary}) + + check[0].should eq(0) + check[1].should eq("Amber agent setup is ready.\n") + end + end + + it "fails coverage when amber-lsp reports declined" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root, coverage: "amber-lsp: declined unsupported file") + + check = AgentHookSpecHelper.run_hook(fixture, "check") + + check[0].should eq(2) + check[1].should contain("amber-lsp does not cover this project main entry file") + end + end + + it "fails when the Claude plugin is stale" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + File.write(".amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json", %({"name":"amber-lsp","version":"0.9.0","description":"old"}\n)) + + check = AgentHookSpecHelper.run_hook(fixture, "check") + + check[0].should eq(2) + check[1].should contain("The Claude Amber LSP plugin files or settings are missing or out of date") + end + end + + it "fails when the Claude marketplace source is not nested under extraKnownMarketplaces" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + File.write(".claude/settings.json", <<-JSON) + { + "extraKnownMarketplaces": { + "amber": { + "source": "directory", + "path": "./.amber/claude-marketplace" + } + }, + "enabledPlugins": {"amber-lsp@amber": true} + } + JSON + + check = AgentHookSpecHelper.run_hook(fixture, "check") + + check[0].should eq(2) + check[1].should contain("The Claude Amber LSP plugin files or settings are missing or out of date") + end + end + + it "invalidates its cache when the setup record changes" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + first_check = AgentHookSpecHelper.run_hook(fixture, "check") + first_check[0].should eq(0) + first_calls = File.read(fixture.lsp_log).lines.size + setup_manifest = AmberCLI::Agent::AgentSetupManifest.new(AmberCli::VERSION, "1.0.0", "3") + File.write(".amber/agent_setup.json", setup_manifest.to_json + "\n ") + + second_check = AgentHookSpecHelper.run_hook(fixture, "check") + + second_check[0].should eq(0) + File.read(fixture.lsp_log).lines.size.should be > first_calls + end + end + + it "blocks Edit and Write Crystal paths and allows Markdown edits in both setup states" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + payloads = [ + {"tool_name" => "Edit", "tool_input" => {"file_path" => "/project/src/model.cr"}}.to_json, + {"tool_name" => "Write", "tool_input" => {"file_path" => "/project/src/new_model.cr"}}.to_json, + ] + markdown = {"tool_name" => "Edit", "tool_input" => {"file_path" => "/project/README.md"}}.to_json + + payloads.each do |payload| + AgentHookSpecHelper.run_hook(fixture, "pre", payload)[0].should eq(0) + AgentHookSpecHelper.run_hook(fixture, "pre", payload, {"AMBER_LSP_BIN" => "/nonexistent"})[0].should eq(2) + end + AgentHookSpecHelper.run_hook(fixture, "pre", markdown, {"AMBER_LSP_BIN" => "/nonexistent"})[0].should eq(0) + end + end + + it "blocks Codex apply_patch additions and Bash Crystal heredocs while allowing no-write Bash" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + codex_patch = {"tool_name" => "apply_patch", "tool_input" => {"command" => "*** Begin Patch\n*** Add File: src/new_model.cr\n+class NewModel\n+end\n*** End Patch"}}.to_json + bash_heredoc = {"tool_name" => "Bash", "tool_input" => {"command" => "cat > src/new_model.cr <<'EOF'\nclass NewModel\nend\nEOF"}}.to_json + no_write_bash = {"tool_name" => "Bash", "tool_input" => {"command" => "crystal-alpha spec"}}.to_json + + [codex_patch, bash_heredoc].each do |payload| + AgentHookSpecHelper.run_hook(fixture, "pre", payload)[0].should eq(0) + AgentHookSpecHelper.run_hook(fixture, "pre", payload, {"AMBER_LSP_BIN" => "/nonexistent"})[0].should eq(2) + end + AgentHookSpecHelper.run_hook(fixture, "pre", no_write_bash, {"AMBER_LSP_BIN" => "/nonexistent"})[0].should eq(0) + end + end + + it "detects Codex tee writes into Crystal files" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + tee_payload = {"tool_name" => "Bash", "tool_input" => {"command" => "printf 'class NewModel' | tee src/new_model.cr"}}.to_json + + result = AgentHookSpecHelper.run_hook(fixture, "pre", tee_payload, {"AMBER_LSP_BIN" => "/nonexistent"}) + + result[0].should eq(2) + result[2].should contain("The amber-lsp binary was not found") + end + end +end diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook index 7172f13..066ad41 100644 --- a/src/amber_cli/templates/agent/amber-agent-hook +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -4,6 +4,10 @@ set -u project_root=$(CDPATH='' cd "$(dirname "$0")/.." && pwd -P) || exit 2 cd "$project_root" || exit 2 +HOOK_VERSION=3 +DEFAULT_MINIMUM_AMBER_LSP_VERSION=1.0.0 +MAIN_FILE='__AMBER_MAIN__' + compiler_is_stock=0 if [ -n "${CRYSTAL_ALPHA:-}" ]; then compiler=$CRYSTAL_ALPHA @@ -16,20 +20,36 @@ elif command -v crystal >/dev/null 2>&1; then compiler_is_stock=1 echo 'amber-agent-hook: fast rebuilds need crystal-alpha; using stock crystal' >&2 else - echo 'amber-agent-hook: no Crystal compiler found' >&2 - exit 2 + compiler='' fi watch_coordination_is_available() { [ "$compiler_is_stock" -eq 0 ] || return 1 + [ -n "$compiler" ] || return 1 "$compiler" watch status >/dev/null 2>&1 watch_status=$? [ "$watch_status" -eq 0 ] || [ "$watch_status" -eq 2 ] } +resolve_lsp() { + if [ -n "${AMBER_LSP_BIN:-}" ]; then + case "$AMBER_LSP_BIN" in + */*) [ -x "$AMBER_LSP_BIN" ] && { printf '%s\n' "$AMBER_LSP_BIN"; return 0; } ;; + *) command -v "$AMBER_LSP_BIN" 2>/dev/null && return 0 ;; + esac + return 1 + fi + + command -v amber-lsp 2>/dev/null +} + find_lsp() { - configured_lsp='' - if [ -f .lsp.json ]; then + if [ -n "${AMBER_LSP_BIN:-}" ]; then + resolve_lsp && return 0 + return 1 + fi + for lsp_config in .lsp.json .amber/claude-marketplace/amber-lsp/.lsp.json; do + [ -f "$lsp_config" ] || continue configured_lsp=$(awk ' function string_value( character, escaped, result) { result = ""; escaped = 0 @@ -60,10 +80,7 @@ find_lsp() { if (key == "command") { print value; exit } } } - ' .lsp.json) - fi - - if [ -n "$configured_lsp" ]; then + ' "$lsp_config") if [ -x "$configured_lsp" ]; then printf '%s\n' "$configured_lsp" return 0 @@ -71,26 +88,456 @@ find_lsp() { case "$configured_lsp" in amber-lsp) ;; *) - if command -v "$configured_lsp" >/dev/null 2>&1; then + if [ -n "$configured_lsp" ] && command -v "$configured_lsp" >/dev/null 2>&1; then printf '%s\n' "$configured_lsp" return 0 fi ;; esac - fi + done if [ -x bin/amber-lsp ]; then printf '%s\n' bin/amber-lsp return 0 fi if command -v amber-lsp >/dev/null 2>&1; then - printf '%s\n' amber-lsp + command -v amber-lsp return 0 fi return 1 } +file_mtime() { + mtime=$(stat -f '%m' "$1" 2>/dev/null || :) + case "$mtime" in + ''|*[!0-9]*) stat -c '%Y' "$1" 2>/dev/null ;; + *) printf '%s\n' "$mtime" ;; + esac +} + +sha256_file() { + if command -v sha256sum >/dev/null 2>&1; then + sha256sum "$1" 2>/dev/null | awk '{print $1}' + return + fi + if command -v shasum >/dev/null 2>&1; then + shasum -a 256 "$1" 2>/dev/null | awk '{print $1}' + return + fi + return 1 +} + +json_string_value() { + wanted_key=$1 + json_file=$2 + awk -v wanted_key="$wanted_key" ' + function string_value( character, escaped, result) { + result = ""; escaped = 0 + while (position <= length(document)) { + character = substr(document, position++, 1) + if (escaped) { + if (character == "n") result = result "\n" + else if (character == "r") result = result "\r" + else if (character == "t") result = result "\t" + else result = result character + escaped = 0; continue + } + if (character == "\\") { escaped = 1; continue } + if (character == "\"") return result + result = result character + } + return "" + } + { document = document $0 "\n" } + END { + position = 1 + while (position <= length(document)) { + if (substr(document, position++, 1) != "\"") continue + key = string_value() + while (substr(document, position, 1) ~ /[[:space:]]/) position++ + if (substr(document, position++, 1) != ":") continue + while (substr(document, position, 1) ~ /[[:space:]]/) position++ + if (substr(document, position++, 1) != "\"") continue + value = string_value() + if (key == wanted_key) { print value; exit } + } + } + ' "$json_file" +} + +add_check_failure() { + if [ -n "$CHECK_FAILURES" ]; then + CHECK_FAILURES="${CHECK_FAILURES} +$1" + else + CHECK_FAILURES=$1 + fi +} + +version_is_at_least() { + awk -v actual="$1" -v required="$2" ' + function core(value) { sub(/[-+].*$/, "", value); return value } + BEGIN { + actual_version = core(actual) + required_version = core(required) + actual_count = split(actual_version, actual_parts, ".") + required_count = split(required_version, required_parts, ".") + if (actual_count < 2 || required_count < 2) exit 1 + for (part = 1; part <= 3; part++) { + if ((actual_parts[part] + 0) > (required_parts[part] + 0)) exit 0 + if ((actual_parts[part] + 0) < (required_parts[part] + 0)) exit 1 + } + if (actual ~ /-/ && actual_version == required_version) exit 1 + exit 0 + } + ' +} + +checksum_from_release_file() { + checksum_file=$1 + binary_name=$2 + awk -v binary_name="$binary_name" ' + function basename(path, parts, count) { + count = split(path, parts, "/") + return parts[count] + } + { + digest = "" + filename = "" + for (field = 1; field <= NF; field++) { + value = $field + sub(/^\*/, "", value) + if (length(value) == 64 && value ~ /^[[:xdigit:]]+$/) digest = value + else filename = basename(value) + } + if (digest != "" && filename == binary_name) { print digest; exit } + } + ' "$checksum_file" +} + +claude_marketplace_setting_is_current() { + awk ' + { + brace_text = $0 + open_braces = gsub(/\{/, "", brace_text) + close_braces = gsub(/\}/, "", brace_text) + depth_before = depth + depth_change = open_braces - close_braces + + if (!in_marketplace && /"extraKnownMarketplaces"[[:space:]]*:/) { + depth += depth_change + marketplace_depth = depth + in_marketplace = 1 + next + } + + if (in_marketplace) { + if (!amber_depth && depth_before == marketplace_depth && /"amber"[[:space:]]*:/) { + depth += depth_change + amber_depth = depth + next + } + if (amber_depth && !source_depth && depth_before == amber_depth && /"source"[[:space:]]*:[[:space:]]*\{/) { + depth += depth_change + source_depth = depth + if (/"source"[[:space:]]*:[[:space:]]*"directory"/) has_directory_source = 1 + if (/"path"[[:space:]]*:[[:space:]]*"\.\/\.amber\/claude-marketplace"/) has_marketplace_path = 1 + if (depth <= amber_depth && has_directory_source && has_marketplace_path) found = 1 + if (found) exit 0 + next + } + if (source_depth && depth_before == source_depth) { + if (/"source"[[:space:]]*:[[:space:]]*"directory"/) has_directory_source = 1 + if (/"path"[[:space:]]*:[[:space:]]*"\.\/\.amber\/claude-marketplace"/) has_marketplace_path = 1 + } + + depth += depth_change + if (source_depth && depth < source_depth) { + if (has_directory_source && has_marketplace_path) found = 1 + source_depth = 0 + has_directory_source = 0 + has_marketplace_path = 0 + } + if (amber_depth && depth < amber_depth) { + amber_depth = 0 + source_depth = 0 + has_directory_source = 0 + has_marketplace_path = 0 + } + if (depth < marketplace_depth) in_marketplace = 0 + if (found) exit 0 + next + } + + depth += depth_change + } + END { if (found) exit 0; exit 1 } + ' "$1" +} + +setup_cache_key() { + lsp_path=$1 + lsp_mtime=missing + if [ -n "$lsp_path" ] && [ -f "$lsp_path" ]; then + lsp_mtime=$(file_mtime "$lsp_path" 2>/dev/null || printf unknown) + fi + setup_signature=missing + if [ -f .amber/agent_setup.json ]; then + setup_signature=$(cksum < .amber/agent_setup.json) + fi + dependent_mtimes='' + for dependent_file in \ + "$lsp_path.sha256" \ + "$(dirname "$(dirname "$lsp_path")")/share/amber_cli/checksums.txt" \ + .amber/claude-marketplace/.claude-plugin/marketplace.json \ + .amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json \ + .amber/claude-marketplace/amber-lsp/.lsp.json \ + .claude/settings.json \ + "$MAIN_FILE" \ + shard.yml + do + if [ -f "$dependent_file" ]; then + dependent_mtime=$(file_mtime "$dependent_file" 2>/dev/null || printf unknown) + dependent_mtimes="${dependent_mtimes}${dependent_mtimes:+,}${dependent_file}:${dependent_mtime}" + else + dependent_mtimes="${dependent_mtimes}${dependent_mtimes:+,}${dependent_file}:missing" + fi + done + cache_project=$(printf '%s' "$project_root" | cksum) + cache_identity=$(printf '%s\n%s\n%s\n%s' "$lsp_path" "$lsp_mtime" "$setup_signature" "$dependent_mtimes" | cksum) + CHECK_CACHE_KEY="$cache_project:$cache_identity" +} + +compute_setup_check() { + CHECK_FAILURES='' + CHECK_VERSION='unknown' + lsp=$(resolve_lsp 2>/dev/null || :) + setup_cache_key "$lsp" + cache_dir=${TMPDIR:-/tmp} + cache_file="$cache_dir/amber-agent-hook-$CHECK_CACHE_KEY.cache" + if [ -f "$cache_file" ]; then + cached_key='' + cached_version='unknown' + cached_failures='' + { + IFS= read -r cached_key || : + IFS= read -r cached_version || : + while IFS= read -r cached_failure; do + if [ -n "$cached_failures" ]; then + cached_failures="${cached_failures} +${cached_failure}" + else + cached_failures=$cached_failure + fi + done + } < "$cache_file" + if [ "$cached_key" = "$CHECK_CACHE_KEY" ]; then + CHECK_VERSION=$cached_version + CHECK_FAILURES=$cached_failures + return + fi + fi + + setup_manifest=.amber/agent_setup.json + minimum_version=$DEFAULT_MINIMUM_AMBER_LSP_VERSION + if [ -f "$setup_manifest" ]; then + saved_minimum=$(json_string_value minimum_amber_lsp_version "$setup_manifest") + saved_hook_version=$(json_string_value generated_hook_version "$setup_manifest") + if [ -n "$saved_minimum" ]; then minimum_version=$saved_minimum; fi + if [ "$saved_hook_version" != "$HOOK_VERSION" ]; then + add_check_failure "The generated agent setup metadata is out of date; run \`amber setup:agent\`." + fi + else + add_check_failure "The project agent setup metadata is missing; run \`amber setup:agent\`." + fi + + binary_is_verified=0 + if [ -z "$lsp" ] || [ ! -x "$lsp" ]; then + requested_binary=${AMBER_LSP_BIN:-amber-lsp} + add_check_failure "The amber-lsp binary was not found at $requested_binary or on PATH." + else + version_output=$("$lsp" --version 2>&1) + version_status=$? + CHECK_VERSION=$(printf '%s\n' "$version_output" | awk '$1 == "amber-lsp" {print $2; exit}') + if [ "$version_status" -ne 0 ] || [ -z "$CHECK_VERSION" ]; then + add_check_failure 'The amber-lsp binary did not report a readable version; install a supported release.' + CHECK_VERSION=unknown + elif ! version_is_at_least "$CHECK_VERSION" "$minimum_version"; then + add_check_failure "The amber-lsp version $CHECK_VERSION is below the required $minimum_version; update amber-lsp." + fi + + if [ -f "$lsp.sha256" ]; then + expected_checksum=$(awk 'NR == 1 { if ($1 == "SHA256") print $NF; else print $1; exit }' "$lsp.sha256") + else + checksum_file="$(dirname "$(dirname "$lsp")")/share/amber_cli/checksums.txt" + expected_checksum='' + if [ -f "$checksum_file" ]; then + expected_checksum=$(checksum_from_release_file "$checksum_file" "$(basename "$lsp")") + fi + fi + actual_checksum=$(sha256_file "$lsp" 2>/dev/null || :) + expected_checksum=$(printf '%s' "$expected_checksum" | tr '[:upper:]' '[:lower:]') + actual_checksum=$(printf '%s' "$actual_checksum" | tr '[:upper:]' '[:lower:]') + case "$expected_checksum" in + *[!0123456789abcdef]*|'') checksum_ok=0 ;; + *) [ "${#expected_checksum}" -eq 64 ] && [ "$expected_checksum" = "$actual_checksum" ] && checksum_ok=1 || checksum_ok=0 ;; + esac + if [ "$checksum_ok" -eq 0 ]; then + add_check_failure 'The amber-lsp checksum is missing or does not match; reinstall its checksum-verified release.' + elif [ "$version_status" -eq 0 ] && [ -n "$CHECK_VERSION" ] && version_is_at_least "$CHECK_VERSION" "$minimum_version"; then + binary_is_verified=1 + fi + fi + + if [ "$binary_is_verified" -eq 1 ]; then + coverage_output=$("$lsp" --check "$MAIN_FILE" 2>&1) + coverage_status=$? + case "$coverage_output" in + *declined*|*failed*) + add_check_failure "amber-lsp does not cover this project main entry file; update the Amber LSP rules and run \`amber setup:agent\` again." + ;; + *) + if [ "$coverage_status" -ne 0 ]; then + add_check_failure "amber-lsp could not confirm coverage for this project main entry file; run \`amber setup:agent\` after fixing the project." + fi + ;; + esac + fi + + marketplace=.amber/claude-marketplace/.claude-plugin/marketplace.json + plugin=.amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json + server=.amber/claude-marketplace/amber-lsp/.lsp.json + settings=.claude/settings.json + if [ ! -f "$marketplace" ] || [ ! -f "$plugin" ] || [ ! -f "$server" ] || [ ! -f "$settings" ] || \ + ! grep -Fq '"name": "amber"' "$marketplace" || \ + ! grep -Fq '"name": "amber-lsp"' "$marketplace" || \ + ! grep -Fq '"source": "./amber-lsp"' "$marketplace" || \ + ! grep -Fq '"name": "amber-lsp"' "$plugin" || \ + ! grep -Fq '"version": "1.0.0"' "$plugin" || \ + ! grep -Fq '"description": "Language server for Amber V2 projects."' "$plugin" || \ + ! grep -Fq '"command": "amber-lsp"' "$server" || \ + ! grep -Fq '".cr": "crystal"' "$server" || \ + ! claude_marketplace_setting_is_current "$settings" || \ + ! grep -Fq '"amber-lsp@amber": true' "$settings"; then + add_check_failure "The Claude Amber LSP plugin files or settings are missing or out of date; run \`amber setup:agent\`." + fi + + cache_temp="$cache_file.$$" + if { + printf '%s\n' "$CHECK_CACHE_KEY" + printf '%s\n' "$CHECK_VERSION" + [ -z "$CHECK_FAILURES" ] || printf '%s\n' "$CHECK_FAILURES" + } > "$cache_temp" 2>/dev/null && mv "$cache_temp" "$cache_file" 2>/dev/null; then + : + else + rm -f "$cache_temp" + fi +} + +run_setup_check() { + compute_setup_check + if [ -n "$CHECK_FAILURES" ]; then + return 2 + fi + return 0 +} + +print_setup_failures() { + if [ -n "$CHECK_FAILURES" ]; then + printf '%s\n' "$CHECK_FAILURES" + fi +} + +print_setup_refusal() { + print_setup_failures + printf '%s\n' "Do not edit Crystal files yet. Tell the user to run \`amber setup:agent\` in this project, then continue." +} + +pre_write_contains_crystal() { + awk ' + function string_value( character, escaped, result) { + result = ""; escaped = 0 + while (position <= length(payload)) { + character = substr(payload, position++, 1) + if (escaped) { + if (character == "n") result = result "\n" + else if (character == "r") result = result "\r" + else if (character == "t") result = result "\t" + else result = result character + escaped = 0; continue + } + if (character == "\\") { escaped = 1; continue } + if (character == "\"") return result + result = result character + } + return "" + } + { payload = payload $0 "\n" } + END { + position = 1 + while (position <= length(payload)) { + if (substr(payload, position++, 1) != "\"") continue + key = string_value() + while (substr(payload, position, 1) ~ /[[:space:]]/) position++ + if (substr(payload, position++, 1) != ":") continue + while (substr(payload, position, 1) ~ /[[:space:]]/) position++ + if (substr(payload, position++, 1) != "\"") continue + value = string_value() + if (key == "tool_name") tool_name = value + else if (key == "file_path") direct_path = value + else if (key == "command") command_text = value + } + + if ((tool_name == "Edit" || tool_name == "Write") && direct_path ~ /[.]cr$/) { + print "crystal-write" + exit + } + if (tool_name == "apply_patch" && index(command_text, "*** Begin Patch") > 0) { + count = split(command_text, patch_lines, "\n") + for (line_number = 1; line_number <= count; line_number++) { + line = patch_lines[line_number] + if (line ~ /^\*\*\* (Update|Add) File: /) { + sub(/^\*\*\* (Update|Add) File: /, "", line) + sub(/\r$/, "", line) + if (line ~ /[.]cr$/) { print "crystal-write"; exit } + } + } + } + if (tool_name == "Bash") { + gsub(/\"/, "", command_text) + gsub(/\047/, "", command_text) + if (command_text ~ /(^|[[:space:];|&])tee[[:space:]][^;|&]*[.]cr([[:space:];|&]|$)/ || + command_text ~ />[^;|&]*[.]cr([[:space:];|&]|$)/) print "crystal-write" + } + } + ' +} + case "${1:-}" in + check) + if run_setup_check; then + printf '%s\n' 'Amber agent setup is ready.' + exit 0 + fi + print_setup_failures + exit 2 + ;; + session) + if run_setup_check; then + printf '%s\n' 'Amber agent setup is OK.' + printf 'amber-lsp version: %s\n' "$CHECK_VERSION" + printf '%s\n' "Before using a library API you are not sure of, run \`amber-lsp lookup 'Type.method'\` (or the LSP tool's workspaceSymbol/hover)." + else + print_setup_refusal + fi + exit 0 + ;; pre) + if [ -n "$(pre_write_contains_crystal)" ]; then + if ! run_setup_check; then + print_setup_refusal >&2 + exit 2 + fi + fi if watch_coordination_is_available; then "$compiler" watch hold status=$? @@ -218,7 +665,7 @@ case "${1:-}" in [ -z "$output" ] || printf '%s\n' "$output" ;; *) - echo 'Usage: bin/amber-agent-hook pre|post|stop' >&2 + echo 'Usage: amber-agent-hook check|session|pre|post|stop' >&2 exit 2 ;; esac From c7131c4d73db28655efeb78d3b6c59227413ce2f Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 22:10:26 -0400 Subject: [PATCH 27/69] fix(lsp): index dependencies in isolated docs workspaces --- .../lookup/build_layered_api_index_spec.cr | 20 +++- .../lookup/build_layered_api_index.cr | 99 ++++++++++++++----- src/amber_lsp/lookup/index_cache.cr | 1 + .../lookup/normalize_crystal_docs.cr | 14 ++- 4 files changed, 106 insertions(+), 28 deletions(-) diff --git a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr index ea0af3d..f28a1d9 100644 --- a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr +++ b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr @@ -8,6 +8,7 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do cache_root = File.join(root, "cache") Dir.mkdir_p(File.join(project_root, "src")) Dir.mkdir_p(File.join(project_root, "lib", "tiny", "src")) + Dir.mkdir_p(File.join(project_root, "lib", "sibling", "src")) Dir.mkdir_p(File.join(project_root, ".amber-lsp", "api")) Dir.mkdir_p(File.join(project_root, "lib", "tiny", ".amber-lsp", "api")) File.write( @@ -16,7 +17,7 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do ) File.write( File.join(project_root, "shard.lock"), - "version: 2.0\nshards:\n tiny:\n git: https://example.test/tiny.git\n version: 1.2.3+git.commit.abcd\n", + "version: 2.0\nshards:\n sibling:\n git: https://example.test/sibling.git\n version: 1.0.0+git.commit.abcd\n tiny:\n git: https://example.test/tiny.git\n version: 1.2.3+git.commit.abcd\n", ) File.write( File.join(project_root, "lib", "tiny", "shard.yml"), @@ -24,7 +25,11 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do ) File.write( File.join(project_root, "lib", "tiny", "src", "tiny.cr"), - "module Tiny\n class Client\n def ping : Bool\n true\n end\n end\nend\n", + "require \"sibling\"\nmodule Tiny\n class Client\n def ping : Bool\n true\n end\n\n def sibling_token : Sibling::Token\n Sibling::Token.new\n end\n end\nend\n", + ) + File.write( + File.join(project_root, "lib", "sibling", "src", "sibling.cr"), + "module Sibling\n class Token\n end\nend\n", ) File.write( File.join(project_root, "src", "lookup_fixture.cr"), @@ -44,7 +49,9 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do second_build = builder.perform project_layer = first_build.find { |cached| cached.layer.try(&.layer_kind) == "project" } - library_layer = first_build.find { |cached| cached.layer.try(&.layer_kind) == "library" } + library_layer = first_build.find do |cached| + cached.layer.try(&.layer_kind) == "library" && cached.layer.try(&.layer_name) == "tiny" + end standard_library_layer = first_build.find { |cached| cached.layer.try(&.layer_kind) == "stdlib" } project_layer.should_not be_nil library_layer.should_not be_nil @@ -62,6 +69,10 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do tiny_client = library_layer.not_nil!.layer.not_nil!.list_of_types.find { |type| type.name == "Tiny::Client" } tiny_client.should_not be_nil tiny_client.not_nil!.list_of_instance_methods.first.name.should eq("ping") + if client = tiny_client + client.list_of_instance_methods.find { |method| method.name == "sibling_token" }.should_not be_nil + client.location_path.should eq(File.join(project_root, "lib", "tiny", "src", "tiny.cr")) + end standard_types = standard_library_layer.not_nil!.layer.not_nil!.list_of_types dir_type = standard_types.find { |type| type.name == "Dir" } @@ -71,8 +82,9 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do dir_type.not_nil!.list_of_class_methods.any? { |method| method.name == "mkdir_p" }.should be_true file_type.not_nil!.list_of_class_methods.any? { |method| method.name == "mkdir_p" }.should be_false - second_build.size.should eq(3) + second_build.size.should eq(4) second_build.each { |cached| cached.freshness.should eq("fresh") } + Dir.children(File.join(cache_root, "docs-workspaces")).should be_empty end end end diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr index 4a016a0..4cdff76 100644 --- a/src/amber_lsp/lookup/build_layered_api_index.cr +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -1,5 +1,7 @@ require "digest/sha256" +require "file_utils" require "process" +require "random/secure" require "yaml" require "./build_crystal_docs_json" @@ -47,6 +49,22 @@ module AmberLSP::Lookup end end + # :nodoc: + struct CrystalDocsWorkspace + getter root_path : String + getter entrypoint : String + getter crystal_cache_path : String + getter source_path_mappings : Hash(String, String) + + def initialize( + @root_path : String, + @entrypoint : String, + @crystal_cache_path : String, + @source_path_mappings : Hash(String, String), + ) + end + end + class BuildLayeredAPIIndex def initialize( @root_path : String, @@ -83,11 +101,6 @@ module AmberLSP::Lookup ) locked_shards = read_locked_shards - if !locked_shards.empty? - project_crystal_path = crystal_path_for_project_libraries(identity) - else - project_crystal_path = nil - end manifest.dependencies.keys.sort.each do |library_name| next if locked_shards.has_key?(library_name) @@ -130,8 +143,6 @@ module AmberLSP::Lookup docs_flags = docs_flags_for(library_name, list_of_docs_flags) layer_key = CalculateLibraryLayerKey.new(library_name, locked_shard.version, docs_flags, identity.version).perform - environment_overrides = {"CRYSTAL_PATH" => project_crystal_path || ""} - list_of_layers << build_or_load_layer( cache, identity, @@ -142,7 +153,7 @@ module AmberLSP::Lookup docs_flags, entrypoint, library_root, - environment_overrides, + {} of String => String, ) end @@ -222,15 +233,30 @@ module AmberLSP::Lookup cached_layer = cache.load_layer(layer_kind, layer_name, layer_key) return cached_layer if cached_layer.freshness == "fresh" || cached_layer.failure_reason + workspace_root : String? = nil begin + docs_working_directory = working_directory + docs_entrypoint = relative_entrypoint(entrypoint, working_directory) + docs_environment_overrides = environment_overrides + source_path_mappings = {} of String => String + + if layer_kind == "library" + workspace = create_library_docs_workspace(source_root, entrypoint) + workspace_root = workspace.root_path + docs_working_directory = workspace.root_path + docs_entrypoint = workspace.entrypoint + docs_environment_overrides = environment_overrides.merge({"CRYSTAL_CACHE_DIR" => workspace.crystal_cache_path}) + source_path_mappings = workspace.source_path_mappings + end + docs_json = BuildCrystalDocsJSON.new( identity.executable_path, - working_directory, - relative_entrypoint(entrypoint, working_directory), + docs_working_directory, + docs_entrypoint, layer_name, "0", docs_flags, - environment_overrides, + docs_environment_overrides, ).perform layer = NormalizeCrystalDocs.new( docs_json, @@ -239,6 +265,8 @@ module AmberLSP::Lookup layer_name, layer_key, docs_flags, + docs_working_directory, + source_path_mappings, ).perform cache.write_layer(layer) CachedAPIIndexLayer.new(layer, "fresh") @@ -250,6 +278,43 @@ module AmberLSP::Lookup end CachedAPIIndexLayer.new(nil, "unavailable", failure_reason, layer_kind, layer_name, layer_key) + ensure + FileUtils.rm_rf(workspace_root) if workspace_root + end + end + + private def create_library_docs_workspace(source_root : String, entrypoint : String) : CrystalDocsWorkspace + source_path = File.expand_path(source_root) + library_root = File.dirname(source_path) + workspace_root = "" + + begin + workspace_parent = File.join(@cache_root, "docs-workspaces") + Dir.mkdir_p(workspace_parent) + workspace_root = File.join(workspace_parent, "#{Process.pid}-#{Random::Secure.hex(12)}") + Dir.mkdir_p(workspace_root) + project_lib_path = File.join(@root_path, "lib") + File.symlink(source_path, File.join(workspace_root, "src")) + File.symlink(project_lib_path, File.join(workspace_root, "lib")) + + crystal_cache_path = File.join(workspace_root, "crystal-cache") + Dir.mkdir_p(crystal_cache_path) + source_relative_entrypoint = entrypoint.sub(source_path + "/", "") + workspace_entrypoint = File.join("src", source_relative_entrypoint) + source_path_mappings = { + File.join(workspace_root, "src") => source_path, + File.join(workspace_root, "lib") => project_lib_path, + } + + CrystalDocsWorkspace.new( + workspace_root, + workspace_entrypoint, + crystal_cache_path, + source_path_mappings, + ) + rescue ex : IO::Error + FileUtils.rm_rf(workspace_root) unless workspace_root.empty? + raise APIIndexBuildError.new("Could not create docs workspace for #{library_root}: #{ex.message}") end end @@ -312,18 +377,6 @@ module AmberLSP::Lookup raise APIIndexBuildError.new("Could not locate the Crystal standard library: #{ex.message}") end - private def crystal_path_for_project_libraries(identity : CrystalAlphaIdentity) : String - output = IO::Memory.new - error_output = IO::Memory.new - status = Process.run(identity.executable_path, ["env", "CRYSTAL_PATH"], output: output, error: error_output) - unless status.success? - raise APIIndexBuildError.new("Could not read crystal-alpha CRYSTAL_PATH: #{error_output.to_s.strip}") - end - "#{output.to_s.strip}:#{File.join(@root_path, "lib")}" - rescue ex : IO::Error - raise APIIndexBuildError.new("Could not read crystal-alpha CRYSTAL_PATH: #{ex.message}") - end - private def find_llvm_config(llvm_version : String) : String? version_parts = llvm_version.split('.') major = version_parts.first? diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr index d4fce40..b25b311 100644 --- a/src/amber_lsp/lookup/index_cache.cr +++ b/src/amber_lsp/lookup/index_cache.cr @@ -53,6 +53,7 @@ module AmberLSP::Lookup def perform : String key_material = [ "library", + "docs-workspace-v1", @library_name, @locked_version_or_commit, @docs_flags.sort.join("\0"), diff --git a/src/amber_lsp/lookup/normalize_crystal_docs.cr b/src/amber_lsp/lookup/normalize_crystal_docs.cr index 2913ebb..07279cc 100644 --- a/src/amber_lsp/lookup/normalize_crystal_docs.cr +++ b/src/amber_lsp/lookup/normalize_crystal_docs.cr @@ -9,6 +9,8 @@ module AmberLSP::Lookup @layer_name : String, @layer_key : String, @docs_flags : Array(String), + @docs_working_directory : String? = nil, + @source_path_mappings : Hash(String, String) = {} of String => String, ) end @@ -103,7 +105,17 @@ module AmberLSP::Lookup end private def source_path(filename : String) : String - return filename if Path[filename].absolute? + base_path = @docs_working_directory || @root_path + expanded_path = Path[filename].absolute? ? File.expand_path(filename) : File.expand_path(filename, base_path) + + @source_path_mappings.each do |workspace_path, source_path| + next unless expanded_path == workspace_path || expanded_path.starts_with?(workspace_path + "/") + + relative_path = expanded_path == workspace_path ? "" : expanded_path[(workspace_path.size + 1)..] + return File.expand_path(relative_path, source_path) + end + + return expanded_path if Path[filename].absolute? File.expand_path(filename, @root_path) end From acb9c048dc581f55e59ae0ffd0dd8ea9c589124f Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:35:56 -0400 Subject: [PATCH 28/69] feat(amber-cli): register LSP through local Claude marketplace --- spec/commands/setup_lsp_spec.cr | 93 +++++++- .../agent/claude_marketplace_settings.cr | 41 ++++ .../agent/install_claude_amber_lsp_plugin.cr | 165 ++++++++++++++ src/amber_cli/commands/setup_lsp.cr | 213 ++---------------- 4 files changed, 308 insertions(+), 204 deletions(-) create mode 100644 src/amber_cli/agent/claude_marketplace_settings.cr create mode 100644 src/amber_cli/agent/install_claude_amber_lsp_plugin.cr diff --git a/spec/commands/setup_lsp_spec.cr b/spec/commands/setup_lsp_spec.cr index 5190fd8..f27f34e 100644 --- a/spec/commands/setup_lsp_spec.cr +++ b/spec/commands/setup_lsp_spec.cr @@ -2,16 +2,97 @@ require "../amber_cli_spec" require "../../src/amber_cli/commands/setup_lsp" describe AmberCLI::Commands::SetupLSPCommand do - it "writes the existing LSP discovery configuration from an Amber project" do + it "writes the exact directory marketplace, plugin manifest, and LSP server configuration" do SpecHelper.within_temp_directory do AmberCLI::Commands::SetupLSPCommand.new("setup:lsp").execute - config = File.read(".lsp.json") - config.should contain(%("amber")) - config.should contain(%("extensionToLanguage")) - config.should contain(%(".cr")) - File.file?(".claude-plugin/plugin.json").should be_true + expected_marketplace = <<-JSON + { + "name": "amber", + "owner": { + "name": "Amber Framework" + }, + "plugins": [ + { + "name": "amber-lsp", + "source": "./amber-lsp" + } + ] + } + JSON + File.read(".amber/claude-marketplace/.claude-plugin/marketplace.json").should eq(expected_marketplace + "\n") + + expected_plugin = <<-JSON + { + "name": "amber-lsp", + "version": "1.0.0", + "description": "Language server for Amber V2 projects." + } + JSON + File.read(".amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json").should eq(expected_plugin + "\n") + + expected_lsp = <<-JSON + { + "amber": { + "command": "amber-lsp", + "extensionToLanguage": { + ".cr": "crystal" + } + } + } + JSON + File.read(".amber/claude-marketplace/amber-lsp/.lsp.json").should eq(expected_lsp + "\n") + File.file?(".lsp.json").should be_false + File.file?(".claude-plugin/plugin.json").should be_false File.file?(".amber-lsp.yml").should be_true + + settings = AmberCLI::Agent::ClaudeMarketplaceSettings.from_json(File.read(".claude/settings.json")) + marketplaces = settings.extra_known_marketplaces || raise("Claude marketplace settings are missing") + amber_marketplace = marketplaces["amber"]? || raise("Amber marketplace is missing") + amber_source = amber_marketplace.source + unless amber_source.is_a?(AmberCLI::Agent::ClaudeMarketplaceDirectorySourceDetails) + raise "Amber marketplace source must be a nested directory source" + end + amber_source.source.should eq("directory") + amber_source.path.should eq("./.amber/claude-marketplace") + amber_marketplace.path.should be_nil + enabled_plugins = settings.enabled_plugins || raise("Claude enabled plugins are missing") + enabled_plugins["amber-lsp@amber"].should be_true + end + end + + it "merges marketplace entries and keeps existing Claude settings and hook data" do + SpecHelper.within_temp_directory do + Dir.mkdir_p(".claude") + File.write(".claude/settings.json", <<-JSON) + { + "permissions": {"allow": ["Read"]}, + "extraKnownMarketplaces": { + "shop": { + "source": {"source": "directory", "path": "./.plugins/shop", "marker": "keep"}, + "marker": "keep" + } + }, + "enabledPlugins": {"shop-plugin@shop": true}, + "hooks": { + "SessionStart": [{"hooks": [{"type": "command", "command": "existing-session-hook"}]}] + } + } + JSON + + AmberCLI::Commands::SetupLSPCommand.new("setup:lsp").execute + + settings_json = File.read(".claude/settings.json") + settings = AmberCLI::Agent::ClaudeMarketplaceSettings.from_json(settings_json) + settings_json.should contain("Read") + settings_json.should contain("marker") + settings_json.should contain("keep") + settings_json.should contain("existing-session-hook") + marketplaces = settings.extra_known_marketplaces || raise("Claude marketplace settings are missing") + marketplaces.keys.sort.should eq(["amber", "shop"]) + enabled_plugins = settings.enabled_plugins || raise("Claude enabled plugins are missing") + enabled_plugins["shop-plugin@shop"].should be_true + enabled_plugins["amber-lsp@amber"].should be_true end end end diff --git a/src/amber_cli/agent/claude_marketplace_settings.cr b/src/amber_cli/agent/claude_marketplace_settings.cr new file mode 100644 index 0000000..43dd89a --- /dev/null +++ b/src/amber_cli/agent/claude_marketplace_settings.cr @@ -0,0 +1,41 @@ +require "json" + +module AmberCLI::Agent + class ClaudeMarketplaceDirectorySource + include JSON::Serializable + include JSON::Serializable::Unmapped + + @[JSON::Field(emit_null: false)] + property source : String | ClaudeMarketplaceDirectorySourceDetails = "directory" + @[JSON::Field(emit_null: false)] + property path : String? = nil + + def initialize(@source : String | ClaudeMarketplaceDirectorySourceDetails = "directory", @path : String? = nil) + end + end + + class ClaudeMarketplaceDirectorySourceDetails + include JSON::Serializable + include JSON::Serializable::Unmapped + + property source : String + property path : String + + def initialize(@source : String, @path : String) + end + end + + # Keeps unrelated Claude project settings while merging the Amber marketplace. + class ClaudeMarketplaceSettings + include JSON::Serializable + include JSON::Serializable::Unmapped + + @[JSON::Field(key: "extraKnownMarketplaces", emit_null: false)] + property extra_known_marketplaces : Hash(String, ClaudeMarketplaceDirectorySource)? = nil + @[JSON::Field(key: "enabledPlugins", emit_null: false)] + property enabled_plugins : Hash(String, Bool)? = nil + + def initialize + end + end +end diff --git a/src/amber_cli/agent/install_claude_amber_lsp_plugin.cr b/src/amber_cli/agent/install_claude_amber_lsp_plugin.cr new file mode 100644 index 0000000..2a5dfef --- /dev/null +++ b/src/amber_cli/agent/install_claude_amber_lsp_plugin.cr @@ -0,0 +1,165 @@ +require "json" +require "./claude_marketplace_settings" + +module AmberCLI::Agent + MINIMUM_AMBER_LSP_VERSION = "1.0.0" + CLAUDE_AMBER_LSP_PLUGIN_VERSION = "1.0.0" + + class ClaudeMarketplaceOwner + include JSON::Serializable + + property name : String + + def initialize(@name : String) + end + end + + class ClaudeMarketplacePlugin + include JSON::Serializable + + property name : String + property source : String + + def initialize(@name : String, @source : String) + end + end + + class ClaudeMarketplaceManifest + include JSON::Serializable + + property name : String + property owner : ClaudeMarketplaceOwner + property plugins : Array(ClaudeMarketplacePlugin) + + def initialize(@name : String, @owner : ClaudeMarketplaceOwner, @plugins : Array(ClaudeMarketplacePlugin)) + end + end + + class ClaudePluginManifest + include JSON::Serializable + + property name : String + property version : String + property description : String + + def initialize(@name : String, @version : String, @description : String) + end + end + + class AmberLSPServerSettings + include JSON::Serializable + + property command : String + @[JSON::Field(key: "extensionToLanguage")] + property extension_to_language : Hash(String, String) + + def initialize(@command : String, @extension_to_language : Hash(String, String)) + end + end + + class ClaudeLSPServerSettings + include JSON::Serializable + + property amber : AmberLSPServerSettings + + def initialize(@amber : AmberLSPServerSettings) + end + end + + # Merges the Amber directory marketplace into the project's Claude settings. + class MergeAmberLSPIntoClaudeSettings + MARKETPLACE_NAME = "amber" + MARKETPLACE_PATH = "./.amber/claude-marketplace" + PLUGIN_NAME = "amber-lsp" + + def initialize(@existing_json : String) + end + + def perform : String + settings = @existing_json.empty? ? ClaudeMarketplaceSettings.new : ClaudeMarketplaceSettings.from_json(@existing_json) + marketplace_sources = settings.extra_known_marketplaces || {} of String => ClaudeMarketplaceDirectorySource + marketplace_source = marketplace_sources[MARKETPLACE_NAME]? || ClaudeMarketplaceDirectorySource.new + case source = marketplace_source.source + when ClaudeMarketplaceDirectorySourceDetails + source.source = "directory" + source.path = MARKETPLACE_PATH + when String + marketplace_source.source = ClaudeMarketplaceDirectorySourceDetails.new("directory", MARKETPLACE_PATH) + end + marketplace_source.path = nil + marketplace_sources[MARKETPLACE_NAME] = marketplace_source + settings.extra_known_marketplaces = marketplace_sources + + enabled_plugins = settings.enabled_plugins || {} of String => Bool + enabled_plugins["#{PLUGIN_NAME}@#{MARKETPLACE_NAME}"] = true + settings.enabled_plugins = enabled_plugins + settings.to_pretty_json + "\n" + end + end + + # Writes the project-local Claude Code marketplace and Amber LSP plugin. + class InstallClaudeAmberLSPPlugin + def perform : Array(String) + list_of_updated_paths = [] of String + write_project_file( + ".amber/claude-marketplace/.claude-plugin/marketplace.json", + marketplace_manifest.to_pretty_json + "\n", + list_of_updated_paths, + ) + write_project_file( + ".amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json", + plugin_manifest.to_pretty_json + "\n", + list_of_updated_paths, + ) + write_project_file( + ".amber/claude-marketplace/amber-lsp/.lsp.json", + lsp_server_settings.to_pretty_json + "\n", + list_of_updated_paths, + ) + update_claude_settings(list_of_updated_paths) + list_of_updated_paths + end + + private def marketplace_manifest : ClaudeMarketplaceManifest + ClaudeMarketplaceManifest.new( + "amber", + ClaudeMarketplaceOwner.new("Amber Framework"), + [ClaudeMarketplacePlugin.new("amber-lsp", "./amber-lsp")], + ) + end + + private def plugin_manifest : ClaudePluginManifest + ClaudePluginManifest.new( + "amber-lsp", + CLAUDE_AMBER_LSP_PLUGIN_VERSION, + "Language server for Amber V2 projects.", + ) + end + + private def lsp_server_settings : ClaudeLSPServerSettings + server = AmberLSPServerSettings.new("amber-lsp", {".cr" => "crystal"}) + ClaudeLSPServerSettings.new(server) + end + + private def update_claude_settings(list_of_updated_paths : Array(String)) : Nil + path = ".claude/settings.json" + existing_json = File.file?(path) ? File.read(path) : "" + merged_json = MergeAmberLSPIntoClaudeSettings.new(existing_json).perform + return if existing_json == merged_json + + Dir.mkdir_p(File.dirname(path)) + File.write(path, merged_json) + list_of_updated_paths << path + end + + private def write_project_file(path : String, content : String, list_of_updated_paths : Array(String)) : Nil + if File.file?(path) && File.read(path) == content + return + end + + Dir.mkdir_p(File.dirname(path)) + File.write(path, content) + list_of_updated_paths << path + end + end +end diff --git a/src/amber_cli/commands/setup_lsp.cr b/src/amber_cli/commands/setup_lsp.cr index 77bcccc..3631484 100644 --- a/src/amber_cli/commands/setup_lsp.cr +++ b/src/amber_cli/commands/setup_lsp.cr @@ -1,217 +1,35 @@ require "../core/base_command" -require "../agent/resolve_compiler_for_agent_loop" +require "../agent/install_claude_amber_lsp_plugin" -# The `setup:lsp` command configures the Amber LSP server for Claude Code -# integration in an Amber project directory. -# -# ## Usage -# ``` -# amber setup: lsp [OPTIONS] -# ``` -# -# ## What It Does -# 1. Resolves or builds the `amber-lsp` binary -# 2. Creates `.lsp.json` for Claude Code LSP server discovery -# 3. Creates `.claude-plugin/plugin.json` as the plugin manifest -# 4. Creates `.amber-lsp.yml` with default rule configuration -# -# ## Examples -# ``` -# amber setup:lsp -# amber setup:lsp --binary-path=/usr/local/bin/amber-lsp -# amber setup:lsp --skip-build -# ``` module AmberCLI::Commands + # Registers the Amber LSP plugin in a project's Claude Code marketplace. class SetupLSPCommand < AmberCLI::Core::BaseCommand - getter binary_path_option : String? = nil - getter is_skip_build : Bool = false - def help_description : String <<-HELP - Set up the Amber LSP server for Claude Code integration - - Usage: amber setup:lsp [OPTIONS] + Set up the Amber LSP plugin for Claude Code - This command: - 1. Builds the amber-lsp binary (if needed) - 2. Creates .lsp.json for Claude Code LSP discovery - 3. Creates .claude-plugin/plugin.json - 4. Creates .amber-lsp.yml with default configuration + Usage: amber setup:lsp - Options: - --binary-path=PATH Path to pre-built amber-lsp binary - --skip-build Skip building the binary (assume it's on PATH) + This command writes a project-local directory marketplace, the Amber LSP + plugin manifest, and the plugin's LSP server configuration. It merges the + marketplace registration into .claude/settings.json and preserves other + settings. HELP end def setup_command_options - option_parser.separator "" - option_parser.separator "Options:" - - option_parser.on("--binary-path=PATH", "Path to pre-built amber-lsp binary") do |path| - @binary_path_option = path - end - - option_parser.on("--skip-build", "Skip building the binary (assume it's on PATH)") do - @is_skip_build = true - end end def execute - info "Setting up Amber LSP for Claude Code..." - - binary_path = resolve_binary_path - - create_lsp_json(binary_path) - create_plugin_json + info "Setting up the Amber LSP Claude plugin..." + updated_paths = AmberCLI::Agent::InstallClaudeAmberLSPPlugin.new.perform + updated_paths.each { |path| info "Updated: #{path}" } create_default_config - - success "Amber LSP setup complete!" - puts "" - info "Files created:" - info " .lsp.json - LSP server configuration" - info " .claude-plugin/plugin.json - Claude Code plugin manifest" - info " .amber-lsp.yml - Rule configuration (customize as needed)" - puts "" - info "The LSP will activate automatically when you start Claude Code in this directory." - end - - private def resolve_binary_path : String - # 1. Check --binary-path option first - if path = binary_path_option - unless File.exists?(path) - error "Specified binary not found: #{path}" - exit(1) - end - return File.expand_path(path) - end - - # 2. Check if amber-lsp is on PATH - if found = Process.find_executable("amber-lsp") - info "Found amber-lsp on PATH: #{found}" - return found - end - - # 3. Check if we're in the amber_cli project and binary exists - cli_project_root = find_cli_project_root - if cli_project_root - existing_binary = File.join(cli_project_root, "bin", "amber-lsp") - if File.exists?(existing_binary) - info "Found amber-lsp binary: #{existing_binary}" - return existing_binary - end - end - - # 4. If --skip-build is set, use bare command name - if is_skip_build - warning "Skipping build. Assuming 'amber-lsp' is available on PATH at runtime." - return "amber-lsp" - end - - # 5. Try to build it if source is available - if cli_project_root && File.exists?(File.join(cli_project_root, "src", "amber_lsp.cr")) - build_binary(cli_project_root) - else - error "Could not find or build amber-lsp binary." - error "Options:" - error " 1. Run this command from the amber_cli project directory" - error " 2. Use --binary-path=PATH to specify an existing binary" - error " 3. Use --skip-build to assume amber-lsp is on PATH" - exit(1) - end - end - - private def find_cli_project_root : String? - # Check if the current directory is the amber_cli project - if File.exists?("src/amber_lsp.cr") - return Dir.current - end - - nil - end - - private def build_binary(cli_project_root : String) : String - binary_path = File.join(cli_project_root, "bin", "amber-lsp") - source_path = File.join(cli_project_root, "src", "amber_lsp.cr") - - info "Building amber-lsp from source..." - info " Source: #{source_path}" - info " Output: #{binary_path}" - - Dir.mkdir_p(File.join(cli_project_root, "bin")) unless Dir.exists?(File.join(cli_project_root, "bin")) - - resolver = AmberCLI::Agent::ResolveCompilerForAgentLoop.new( - ENV["CRYSTAL_ALPHA"]?, ->(command : String) { Process.find_executable(command) } - ) - compiler = resolver.perform - unless compiler - error "No Crystal compiler found (CRYSTAL_ALPHA, crystal-alpha, acrystal, crystal)" - exit(1) - end - warning "Fast rebuilds need crystal-alpha; using stock crystal." if resolver.stock_compiler? - - process = Process.run( - compiler, - ["build", source_path, "-o", binary_path, "--release"], - output: Process::Redirect::Inherit, - error: Process::Redirect::Inherit - ) - - unless process.success? - error "Failed to build amber-lsp binary" - exit(1) - end - - success "Built amber-lsp successfully" - binary_path - end - - private def create_lsp_json(binary_path : String) - lsp_config = { - "amber" => { - "command" => binary_path, - "args" => [] of String, - "extensionToLanguage" => { - ".cr" => "crystal", - }, - "transport" => "stdio", - "restartOnCrash" => true, - "maxRestarts" => 3, - }, - } - - path = ".lsp.json" - if File.exists?(path) - warning "Overwriting existing #{path}" - end - File.write(path, lsp_config.to_pretty_json + "\n") - info "Created: #{path}" - end - - private def create_plugin_json - dir = ".claude-plugin" - Dir.mkdir_p(dir) unless Dir.exists?(dir) - - plugin_config = { - "name" => "amber-framework-lsp", - "version" => "1.0.0", - "description" => "Convention diagnostics for Amber V2 web framework projects.", - "author" => { - "name" => "Amber Framework", - }, - "homepage" => "https://github.com/amberframework/amber", - "lspServers" => "./.lsp.json", - } - - path = File.join(dir, "plugin.json") - if File.exists?(path) - warning "Overwriting existing #{path}" - end - File.write(path, plugin_config.to_pretty_json + "\n") - info "Created: #{path}" + success "Amber LSP plugin setup complete." + info "Run `amber setup:agent` to install Claude Code and Codex hooks." end - private def create_default_config + private def create_default_config : Nil path = ".amber-lsp.yml" if File.exists?(path) warning "Skipped (exists): #{path} — remove it first to regenerate" @@ -247,10 +65,9 @@ module AmberCLI::Commands YAML File.write(path, content) - info "Created: #{path}" + info "Updated: #{path}" end end end -# Register the command AmberCLI::Core::CommandRegistry.register("setup:lsp", ["lsp"], AmberCLI::Commands::SetupLSPCommand) From f46934bc287a1f5befc50b4886f42e4e294ab241 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:36:13 -0400 Subject: [PATCH 29/69] feat(amber-cli): enforce agent setup in project hooks --- spec/commands/agent_hook_spec.cr | 16 ++- spec/commands/setup_agent_spec.cr | 76 +++++++++-- .../agent/find_agent_hook_ignore_rules.cr | 93 +++++++++++++ src/amber_cli/agent/hook_settings.cr | 126 ++++++++++++++++++ src/amber_cli/commands/setup_agent.cr | 119 +++++------------ 5 files changed, 330 insertions(+), 100 deletions(-) create mode 100644 src/amber_cli/agent/find_agent_hook_ignore_rules.cr create mode 100644 src/amber_cli/agent/hook_settings.cr diff --git a/spec/commands/agent_hook_spec.cr b/spec/commands/agent_hook_spec.cr index 9c36bd4..c760495 100644 --- a/spec/commands/agent_hook_spec.cr +++ b/spec/commands/agent_hook_spec.cr @@ -7,7 +7,6 @@ record AgentHookFixture, project_root : String, hook_path : String, lsp_path : S module AgentHookSpecHelper def self.create_ready_project(project_root : String, version : String = "1.0.0", coverage : String = "amber-lsp: covered no diagnostics", coverage_exit : String = "0") : AgentHookFixture Dir.mkdir_p("src") - Dir.mkdir_p("bin") Dir.mkdir_p(".amber/claude-marketplace/.claude-plugin") Dir.mkdir_p(".amber/claude-marketplace/amber-lsp/.claude-plugin") Dir.mkdir_p(".claude") @@ -53,7 +52,7 @@ module AgentHookSpecHelper JSON script = AmberCLI::Commands::SetupAgentCommand::AGENT_HOOK_SCRIPT.gsub("__AMBER_MAIN__", "src/app.cr") - hook_path = File.join(project_root, "bin/amber-agent-hook") + hook_path = File.join(project_root, ".amber/amber-agent-hook") File.write(hook_path, script) File.chmod(hook_path, 0o755) @@ -112,6 +111,19 @@ module AgentHookSpecHelper end describe "amber-agent-hook readiness and preflight" do + it "prints the installed version and API lookup guidance at SessionStart" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + + session = AgentHookSpecHelper.run_hook(fixture, "session") + + session[0].should eq(0) + session[1].should contain("Amber agent setup is OK.") + session[1].should contain("amber-lsp version: 1.0.0") + session[1].should contain("Before using a library API you are not sure of, run `amber-lsp lookup 'Type.method'` (or the LSP tool's workspaceSymbol/hover).") + end + end + it "checks a verified binary and project coverage once, then serves the cached result under 300 ms" do SpecHelper.within_temp_directory do |project_root| fixture = AgentHookSpecHelper.create_ready_project(project_root) diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr index c21c632..b8139a7 100644 --- a/spec/commands/setup_agent_spec.cr +++ b/spec/commands/setup_agent_spec.cr @@ -3,10 +3,15 @@ require "../../src/amber_cli/commands/setup_agent" class RecordSetupAgentMessages < AmberCLI::Commands::SetupAgentCommand getter list_of_info_messages : Array(String) = [] of String + getter list_of_warning_messages : Array(String) = [] of String protected def info(message : String) @list_of_info_messages << message end + + protected def warning(message : String) + @list_of_warning_messages << message + end end describe "amber setup:agent" do @@ -47,9 +52,18 @@ describe "amber setup:agent" do first.should contain("bin/start-hook") first.should contain(%("extra": "stay")) first.should contain(%("timeout": 42)) - ["pre", "post", "stop"].each do |event| - first.scan(/bin\/amber-agent-hook #{event}/).size.should eq(1) + ["session", "pre", "post", "stop"].each do |event| + first.scan(/\.amber\/amber-agent-hook #{event}/).size.should eq(1) end + first.should contain(%("Edit|Write|Bash")) + end + + it "uses Codex tool names in its pre-edit matcher" do + settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", false).perform + + settings.should contain(%("apply_patch|Bash")) + settings.should contain(".amber/amber-agent-hook session") + settings.should contain(".amber/amber-agent-hook pre") end it "installs the loop into a project without changing existing instructions twice" do @@ -69,26 +83,63 @@ describe "amber setup:agent" do first_claude = File.read(".claude/settings.json") first_codex = File.read(".codex/hooks.json") first_instructions = File.read("CLAUDE.md") - first_script = File.read("bin/amber-agent-hook") + first_script = File.read(".amber/amber-agent-hook") command.execute File.read(".claude/settings.json").should eq(first_claude) File.read(".codex/hooks.json").should eq(first_codex) first_claude.should contain(%("allow": [)) + first_claude.should contain(".amber/amber-agent-hook session") first_codex.should contain("keep this") - first_codex.should contain(%("SessionStart": [])) + first_codex.should contain(".amber/amber-agent-hook session") + first_codex.should contain(%("apply_patch|Bash")) File.read("CLAUDE.md").should eq(first_instructions) File.read("AGENTS.md").scan(/amber-agent-loop:start/).size.should eq(1) first_instructions.should contain("# Existing Claude instructions") first_instructions.should contain("crystal-alpha spec --affected") + first_instructions.should contain("amber-lsp lookup 'Type.method'") + first_instructions.should contain("run `amber setup:agent` before editing") first_script.should contain("build --no-codegen 'src/custom_entry.cr'") setup_manifest = AmberCLI::Agent::AgentSetupManifest.from_json(File.read(".amber/agent_setup.json")) setup_manifest.amber_cli_version.should eq(AmberCli::VERSION) setup_manifest.minimum_amber_lsp_version.should eq("1.0.0") - setup_manifest.generated_hook_version.should eq("2") - File.file?(".lsp.json").should be_true - File.info("bin/amber-agent-hook").permissions.to_i.&(0o111).should_not eq(0) - command.list_of_info_messages.count("Updated: bin/amber-agent-hook").should eq(1) + setup_manifest.generated_hook_version.should eq("3") + File.file?(".amber/claude-marketplace/amber-lsp/.lsp.json").should be_true + File.file?(".lsp.json").should be_false + File.info(".amber/amber-agent-hook").permissions.to_i.&(0o111).should_not eq(0) + command.list_of_info_messages.count("Updated: .amber/amber-agent-hook").should eq(1) + end + end + + it "reports ignore rules that keep tracked hook settings out of worktrees" do + SpecHelper.within_temp_directory do + File.write("shard.yml", "name: my_app\n") + Dir.mkdir_p("src") + File.write("src/my_app.cr", "puts :ok\n") + Dir.mkdir_p(".git/info") + File.write(".gitignore", "/.claude/\n") + File.write(".git/info/exclude", ".codex/\n") + + command = RecordSetupAgentMessages.new("setup:agent") + command.execute + + command.list_of_warning_messages.size.should eq(2) + command.list_of_warning_messages.should contain("Git ignore rule .git/info/exclude:1:.codex/ matches .codex/hooks.json; agent worktrees will run without those hooks.") + command.list_of_warning_messages.should contain("Git ignore rule .gitignore:1:/.claude/ matches .claude/settings.json; agent worktrees will run without those hooks.") + end + end + + it "honors a later ignore negation for hook settings" do + SpecHelper.within_temp_directory do + File.write("shard.yml", "name: my_app\n") + Dir.mkdir_p("src") + File.write("src/my_app.cr", "puts :ok\n") + File.write(".gitignore", "/.claude/\n!/.claude/\n") + + command = RecordSetupAgentMessages.new("setup:agent") + command.execute + + command.list_of_warning_messages.should_not contain("Git ignore rule .gitignore:1:/.claude/") end end @@ -111,7 +162,7 @@ describe "amber setup:agent" do File.chmod(lsp, 0o755) File.write(".lsp.json", {"amber" => {"command" => lsp}}.to_pretty_json + "\n") path = "#{tools}:/usr/bin:/bin" - hook = File.join(project, "bin/amber-agent-hook") + hook = File.join(project, ".amber/amber-agent-hook") run_hook = ->(event : String, payload : String, watch_exit : String, fallback_exit : String) do output = IO::Memory.new @@ -196,7 +247,7 @@ describe "amber setup:agent" do AmberCLI::Commands::SetupAgentCommand.new("setup:agent").execute - File.read("bin/amber-agent-hook").should contain("build --no-codegen 'src/fallback_app.cr'") + File.read(".amber/amber-agent-hook").should contain("build --no-codegen 'src/fallback_app.cr'") end end @@ -235,7 +286,7 @@ describe "amber setup:agent" do log = File.join(project, "calls.log") File.write(compiler, "#!/bin/sh\nprintf 'compiler %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\ncase \"$1\" in\n watch) echo 'Usage: crystal watch [options] [programfile]'; exit 1 ;;\n build) exit 0 ;;\nesac\n") File.chmod(compiler, 0o755) - hook = File.join(project, "bin/amber-agent-hook") + hook = File.join(project, ".amber/amber-agent-hook") environment = {"PATH" => "#{tools}:/usr/bin:/bin", "TEST_COMMAND_LOG" => log} pre_output = IO::Memory.new @@ -272,7 +323,7 @@ describe "amber setup:agent" do log = File.join(project, "calls.log") File.write(compiler, "#!/bin/sh\nprintf 'compiler %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\nif [ \"${TEST_FORMAT_EXIT:-0}\" -ne 0 ]; then\n echo 'format failed'\n exit \"$TEST_FORMAT_EXIT\"\nfi\n") File.chmod(compiler, 0o755) - hook = File.join(project, "bin/amber-agent-hook") + hook = File.join(project, ".amber/amber-agent-hook") payload = {"tool_input" => {"file_path" => "src/my_app.cr"}}.to_json File.write("src/other.cr", "puts :other\n") patch = "*** Begin Patch\n*** Update File: src/my_app.cr\n*** Update File: src/other.cr\n*** End Patch" @@ -286,6 +337,7 @@ describe "amber setup:agent" do File.read(log).scan(/compiler tool format /).size.should eq(2) project_lsp = File.join(project, "bin/amber-lsp") + Dir.mkdir_p("bin") File.write(project_lsp, "#!/bin/sh\nprintf 'project-lsp %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\n") File.chmod(project_lsp, 0o755) errors = IO::Memory.new diff --git a/src/amber_cli/agent/find_agent_hook_ignore_rules.cr b/src/amber_cli/agent/find_agent_hook_ignore_rules.cr new file mode 100644 index 0000000..b598dff --- /dev/null +++ b/src/amber_cli/agent/find_agent_hook_ignore_rules.cr @@ -0,0 +1,93 @@ +module AmberCLI::Agent + record IgnoredAgentHookSetting, path : String, matching_line : String + + # Finds ignore rules that keep an agent hook settings file out of Git worktrees. + class FindAgentHookIgnoreRules + LIST_OF_SETTINGS_PATHS = [".claude/settings.json", ".codex/hooks.json"] + + def initialize(@project_root : String) + end + + def perform : Array(IgnoredAgentHookSetting) + git_matches = read_git_matches + return git_matches if git_matches + + read_ignore_files + end + + private def read_git_matches : Array(IgnoredAgentHookSetting)? + output = IO::Memory.new + errors = IO::Memory.new + status = Process.run( + "git", + ["check-ignore", "-v", "--no-index", "--", *LIST_OF_SETTINGS_PATHS], + chdir: @project_root, + output: output, + error: errors, + ) + return [] of IgnoredAgentHookSetting if status.exit_code == 1 && output.to_s.empty? + return nil unless status.success? + + output.to_s.lines.compact_map do |result_line| + fields = result_line.split('\t') + next unless matching_path = fields.last? + next unless LIST_OF_SETTINGS_PATHS.includes?(matching_path) + rule_line = fields.first? || result_line + IgnoredAgentHookSetting.new(matching_path, rule_line) + end + rescue ex : File::NotFoundError + nil + end + + private def read_ignore_files : Array(IgnoredAgentHookSetting) + list_of_rules = {} of String => String? + [ + {File.join(@project_root, ".git/info/exclude"), ".git/info/exclude"}, + {File.join(@project_root, ".gitignore"), ".gitignore"}, + ].each do |file_path, display_path| + next unless File.file?(file_path) + + line_number = 0 + File.each_line(file_path) do |raw_line| + line_number += 1 + rule = raw_line.rstrip + next if rule.empty? || rule.starts_with?('#') + + is_negation = rule.starts_with?('!') + pattern = is_negation ? rule.lchop('!') : rule + LIST_OF_SETTINGS_PATHS.each do |settings_path| + directory = File.dirname(settings_path) + next unless pattern.includes?(directory) + next unless may_match_agent_settings?(pattern, settings_path, directory) + + if is_negation + list_of_rules[directory] = nil + else + list_of_rules[directory] = "#{display_path}:#{line_number}:#{rule}" + end + end + end + end + + LIST_OF_SETTINGS_PATHS.compact_map do |settings_path| + directory = File.dirname(settings_path) + if matching_line = list_of_rules[directory]? + IgnoredAgentHookSetting.new(settings_path, matching_line) + end + end + end + + private def may_match_agent_settings?(pattern : String, settings_path : String, directory : String) : Bool + normalized_pattern = pattern.lchop('/') + normalized_pattern = normalized_pattern[3..] if normalized_pattern.starts_with?("**/") + if normalized_pattern.ends_with?('/') + normalized_pattern = normalized_pattern[0...normalized_pattern.size - 1] + end + return true if normalized_pattern == directory || normalized_pattern == settings_path + return true if normalized_pattern == "#{directory}/**" + return false if normalized_pattern.includes?('/') + + normalized_pattern == directory || normalized_pattern == "#{directory}*" + end + end +end diff --git a/src/amber_cli/agent/hook_settings.cr b/src/amber_cli/agent/hook_settings.cr new file mode 100644 index 0000000..577f86e --- /dev/null +++ b/src/amber_cli/agent/hook_settings.cr @@ -0,0 +1,126 @@ +require "json" + +module AmberCLI::Agent + class HookHandler + include JSON::Serializable + include JSON::Serializable::Unmapped + + property type : String? = nil + property command : String? = nil + + def initialize(@type : String?, @command : String?) + end + end + + class HookGroup + include JSON::Serializable + include JSON::Serializable::Unmapped + + @[JSON::Field(emit_null: false)] + property matcher : String? = nil + property hooks : Array(HookHandler) = [] of HookHandler + + def initialize(@matcher : String?, @hooks : Array(HookHandler)) + end + end + + class HookEvents + include JSON::Serializable + include JSON::Serializable::Unmapped + + @[JSON::Field(key: "SessionStart")] + property session_start : Array(HookGroup) = [] of HookGroup + @[JSON::Field(key: "PreToolUse")] + property pre_tool_use : Array(HookGroup) = [] of HookGroup + @[JSON::Field(key: "PostToolUse")] + property post_tool_use : Array(HookGroup) = [] of HookGroup + @[JSON::Field(key: "Stop")] + property stop : Array(HookGroup) = [] of HookGroup + + def initialize + end + end + + class ClaudeMarketplaceSource + include JSON::Serializable + include JSON::Serializable::Unmapped + + @[JSON::Field(emit_null: false)] + property source : String | ClaudeMarketplaceSourceDetails = "directory" + @[JSON::Field(emit_null: false)] + property path : String? = nil + + def initialize(@source : String | ClaudeMarketplaceSourceDetails = "directory", @path : String? = nil) + end + end + + class ClaudeMarketplaceSourceDetails + include JSON::Serializable + include JSON::Serializable::Unmapped + + property source : String + property path : String + + def initialize(@source : String, @path : String) + end + end + + class HookSettings + include JSON::Serializable + include JSON::Serializable::Unmapped + + @[JSON::Field(emit_null: false)] + property hooks : HookEvents? = nil + @[JSON::Field(key: "extraKnownMarketplaces", emit_null: false)] + property extra_known_marketplaces : Hash(String, ClaudeMarketplaceSource)? = nil + @[JSON::Field(key: "enabledPlugins", emit_null: false)] + property enabled_plugins : Hash(String, Bool)? = nil + + def initialize + end + end + + # When an agent loop is installed, retain unrelated hook groups and settings. + class MergeAgentHooksIntoSettings + CLAUDE_PRE_MATCHER = "Edit|Write|Bash" + CODEX_PRE_MATCHER = "apply_patch|Bash" + POST_MATCHER = "Edit|Write|MultiEdit|NotebookEdit" + + def initialize(@existing_json : String, @is_claude_settings : Bool = true) + end + + def perform : String + settings = @existing_json.empty? ? HookSettings.new : HookSettings.from_json(@existing_json) + hooks = settings.hooks || HookEvents.new + pre_matcher = @is_claude_settings ? CLAUDE_PRE_MATCHER : CODEX_PRE_MATCHER + remove_stale_pre_hook(hooks.pre_tool_use, pre_matcher, ".amber/amber-agent-hook pre") + add_hook(hooks.session_start, nil, ".amber/amber-agent-hook session") + add_hook(hooks.pre_tool_use, pre_matcher, ".amber/amber-agent-hook pre") + add_hook(hooks.post_tool_use, POST_MATCHER, ".amber/amber-agent-hook post") + add_hook(hooks.stop, nil, ".amber/amber-agent-hook stop") + settings.hooks = hooks + settings.to_pretty_json + "\n" + end + + private def remove_stale_pre_hook(groups : Array(HookGroup), matcher : String, command : String) : Nil + groups.each do |group| + next if group.matcher == matcher + + group.hooks = group.hooks.reject do |handler| + handler.type == "command" && handler.command == command + end + end + end + + private def add_hook(groups : Array(HookGroup), matcher : String?, command : String) : Nil + group = groups.find { |candidate| candidate.matcher == matcher } + unless group + group = HookGroup.new(matcher, [] of HookHandler) + groups << group + end + return if group.hooks.any? { |handler| handler.type == "command" && handler.command == command } + + group.hooks << HookHandler.new("command", command) + end + end +end diff --git a/src/amber_cli/commands/setup_agent.cr b/src/amber_cli/commands/setup_agent.cr index 3286006..6cce7cd 100644 --- a/src/amber_cli/commands/setup_agent.cr +++ b/src/amber_cli/commands/setup_agent.cr @@ -1,6 +1,9 @@ require "json" require "yaml" require "../core/base_command" +require "../agent/hook_settings" +require "../agent/resolve_compiler_for_agent_loop" +require "../agent/find_agent_hook_ignore_rules" require "../../version" require "./setup_lsp" @@ -19,91 +22,16 @@ module AmberCLI::Agent ) end end - - class HookHandler - include JSON::Serializable - include JSON::Serializable::Unmapped - - property type : String? = nil - property command : String? = nil - - def initialize(@type : String?, @command : String?) - end - end - - class HookGroup - include JSON::Serializable - include JSON::Serializable::Unmapped - - @[JSON::Field(emit_null: false)] - property matcher : String? = nil - property hooks : Array(HookHandler) = [] of HookHandler - - def initialize(@matcher : String?, @hooks : Array(HookHandler)) - end - end - - class HookEvents - include JSON::Serializable - include JSON::Serializable::Unmapped - - @[JSON::Field(key: "PreToolUse")] - property pre_tool_use : Array(HookGroup) = [] of HookGroup - @[JSON::Field(key: "PostToolUse")] - property post_tool_use : Array(HookGroup) = [] of HookGroup - @[JSON::Field(key: "Stop")] - property stop : Array(HookGroup) = [] of HookGroup - - def initialize - end - end - - class HookSettings - include JSON::Serializable - include JSON::Serializable::Unmapped - - property hooks : HookEvents = HookEvents.new - - def initialize - end - end - - # When an agent loop is installed, retain unrelated hook groups and settings. - class MergeAgentHooksIntoSettings - MATCHER = "Edit|Write|MultiEdit|NotebookEdit" - - def initialize(@existing_json : String) - end - - def perform : String - settings = @existing_json.empty? ? HookSettings.new : HookSettings.from_json(@existing_json) - add_hook(settings.hooks.pre_tool_use, MATCHER, "bin/amber-agent-hook pre") - add_hook(settings.hooks.post_tool_use, MATCHER, "bin/amber-agent-hook post") - add_hook(settings.hooks.stop, nil, "bin/amber-agent-hook stop") - settings.to_pretty_json + "\n" - end - - private def add_hook(groups : Array(HookGroup), matcher : String?, command : String) : Nil - group = groups.find { |candidate| candidate.matcher == matcher } - unless group - group = HookGroup.new(matcher, [] of HookHandler) - groups << group - end - return if group.hooks.any? { |handler| handler.type == "command" && handler.command == command } - - group.hooks << HookHandler.new("command", command) - end - end end module AmberCLI::Commands # Installs the optional Claude Code and Codex feedback loop into an Amber V2 app. class SetupAgentCommand < AmberCLI::Core::BaseCommand - AGENT_HOOK_SCRIPT = {{ read_file("#{__DIR__}/../templates/agent/amber-agent-hook") }} - MINIMUM_AMBER_LSP_VERSION = "1.0.0" - GENERATED_HOOK_VERSION = "2" - DOCUMENT_START = "" - DOCUMENT_END = "" + AGENT_HOOK_SCRIPT = {{ read_file("#{__DIR__}/../templates/agent/amber-agent-hook") }} + MINIMUM_AMBER_LSP_VERSION = AmberCLI::Agent::MINIMUM_AMBER_LSP_VERSION + GENERATED_HOOK_VERSION = "3" + DOCUMENT_START = "" + DOCUMENT_END = "" def help_description : String "Set up the Claude Code and Codex agent loop in an Amber V2 project" @@ -115,11 +43,12 @@ module AmberCLI::Commands def execute main_file = find_project_main_file SetupLSPCommand.new("setup:lsp").execute - [".claude/settings.json", ".codex/hooks.json"].each do |path| - write_merged_hooks(path) + [{".claude/settings.json", true}, {".codex/hooks.json", false}].each do |path, is_claude_settings| + write_merged_hooks(path, is_claude_settings) end write_hook_script(main_file) write_setup_manifest + warn_about_ignored_agent_hook_settings ["CLAUDE.md", "AGENTS.md"].each do |path| append_agent_loop_instructions(path) end @@ -148,9 +77,9 @@ module AmberCLI::Commands main_file end - private def write_merged_hooks(path : String) : Nil + private def write_merged_hooks(path : String, is_claude_settings : Bool) : Nil existing_json = File.file?(path) ? File.read(path) : "" - merged_json = AmberCLI::Agent::MergeAgentHooksIntoSettings.new(existing_json).perform + merged_json = AmberCLI::Agent::MergeAgentHooksIntoSettings.new(existing_json, is_claude_settings).perform return if existing_json == merged_json Dir.mkdir_p(File.dirname(path)) @@ -158,10 +87,17 @@ module AmberCLI::Commands info "Updated: #{path}" end + private def warn_about_ignored_agent_hook_settings : Nil + ignored_settings = AmberCLI::Agent::FindAgentHookIgnoreRules.new(Dir.current).perform + ignored_settings.each do |ignored_setting| + warning "Git ignore rule #{ignored_setting.matching_line} matches #{ignored_setting.path}; agent worktrees will run without those hooks." + end + end + private def write_hook_script(main_file : String) : Nil - path = "bin/amber-agent-hook" + path = ".amber/amber-agent-hook" content = AGENT_HOOK_SCRIPT.gsub("__AMBER_MAIN__", main_file) - Dir.mkdir_p("bin") + Dir.mkdir_p(".amber") if !File.file?(path) || File.read(path) != content File.write(path, content) info "Updated: #{path}" @@ -187,6 +123,12 @@ module AmberCLI::Commands if marker_start && marker_end && marker_start < marker_end section = content[marker_start...marker_end] updated_section = section.sub("Use `crystal spec --affected`", "Use `crystal-alpha spec --affected`") + unless updated_section.includes?("amber-lsp lookup 'Type.method'") + updated_section += "\nBefore using a library API you are not sure of, run `amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover." + end + unless updated_section.includes?("stop and tell the user to run `amber setup:agent`") + updated_section += "\nThe SessionStart hook reports whether setup is complete. When setup is missing, stop and tell the user to run `amber setup:agent` before editing Crystal files." + end if updated_section != section File.write(path, content.sub(section, updated_section)) info "Updated: #{path}" @@ -204,6 +146,11 @@ module AmberCLI::Commands Use `crystal-alpha spec --affected` when available. Format Crystal files with `crystal-alpha tool format`. The installed hooks hold the watcher during edits, check each changed file, and build when the agent stops. + Before using a library API you are not sure of, run + `amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover. + The SessionStart hook reports whether setup is complete. When setup is + missing, stop and tell the user to run `amber setup:agent` before editing + Crystal files. #{DOCUMENT_END} MARKDOWN prefix = content.empty? ? "" : content.rstrip + "\n\n" From 2c24e455570dba668ac7c957973fa85dea09c0fe Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:36:27 -0400 Subject: [PATCH 30/69] feat(amber-cli): add agent setup doctor checks --- spec/commands/doctor_spec.cr | 164 +++++++++++++ src/amber_cli.cr | 2 + src/amber_cli/commands/doctor.cr | 410 +++++++++++++++++++++++++++++++ 3 files changed, 576 insertions(+) create mode 100644 spec/commands/doctor_spec.cr create mode 100644 src/amber_cli/commands/doctor.cr diff --git a/spec/commands/doctor_spec.cr b/spec/commands/doctor_spec.cr new file mode 100644 index 0000000..c8df82b --- /dev/null +++ b/spec/commands/doctor_spec.cr @@ -0,0 +1,164 @@ +require "../amber_cli_spec" +require "../../src/amber_cli/commands/doctor" +require "../../src/amber_cli/agent/install_claude_amber_lsp_plugin" + +module DoctorCommandSpecHelper + def self.create_agent_settings(project_root : String) : String + Dir.mkdir_p(File.join(project_root, ".claude")) + Dir.mkdir_p(File.join(project_root, ".codex")) + claude_settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", true).perform + codex_settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", false).perform + File.write(File.join(project_root, ".claude/settings.json"), claude_settings) + File.write(File.join(project_root, ".codex/hooks.json"), codex_settings) + codex_settings + end + + def self.write_executable(path : String, content : String) : Nil + Dir.mkdir_p(File.dirname(path)) + File.write(path, content) + File.chmod(path, 0o755) + end + + def self.restore_environment(name : String, value : String?) : Nil + if value + ENV[name] = value + else + ENV.delete(name) + end + end +end + +describe "amber doctor" do + it "reports project fixes and exits with status 1 when required hooks are missing" do + SpecHelper.within_temp_directory do |project_root| + home_directory = File.join(project_root, "home") + Dir.mkdir_p(home_directory) + + report = AmberCLI::Commands::DoctorCommand.new("doctor", project_root, home_directory, false).perform + output = report.to_s + + report.exit_code.should eq(1) + output.should contain("[FAIL] Claude project hooks are missing from .claude/settings.json; run `amber setup:agent`.") + output.should contain("[FAIL] Codex project hooks are missing from .codex/hooks.json; run `amber setup:agent`.") + output.should contain("[SKIP] API lookup status check skipped by the doctor test harness.") + end + end + + it "reports a ready project with exit status 0 when hooks, trust, plugins, and the API index pass" do + SpecHelper.within_temp_directory do |project_root| + home_directory = File.join(project_root, "home") + tools_directory = File.join(project_root, "tools") + Dir.mkdir_p(home_directory) + codex_settings = DoctorCommandSpecHelper.create_agent_settings(project_root) + + File.write(File.join(home_directory, ".claude.json"), { + "projects" => { + File.realpath(project_root) => {"hasTrustDialogAccepted" => true}, + }, + }.to_json) + DoctorCommandSpecHelper.write_executable( + File.join(tools_directory, "claude"), + "#!/bin/sh\nprintf '%s\\n' '[{\"id\":\"amber-lsp@amber\",\"enabled\":true,\"projectEnabled\":true,\"errors\":[]}]'\n", + ) + DoctorCommandSpecHelper.write_executable(File.join(tools_directory, "codex"), "#!/bin/sh\nexit 0\n") + DoctorCommandSpecHelper.write_executable( + File.join(project_root, ".amber/amber-agent-hook"), + "#!/bin/sh\nprintf '%s\\n' 'Amber agent setup is ready.'\nexit 0\n", + ) + lsp_path = File.join(tools_directory, "amber-lsp") + DoctorCommandSpecHelper.write_executable( + lsp_path, + "#!/bin/sh\n[ \"$1 $2\" = 'lookup --status' ] || exit 2\nprintf '%s\\n' 'Amber API index is fresh'\n", + ) + + codex_settings_path = File.expand_path(File.join(File.realpath(project_root), ".codex/hooks.json")) + trusted_entries = %w[session_start pre_tool_use post_tool_use stop].map do |event| + %([hooks.state."#{codex_settings_path}:#{event}:0:0"]\ntrusted_hash = "sha256:#{"a" * 64}"\n) + end + Dir.mkdir_p(File.join(home_directory, ".codex")) + File.write(File.join(home_directory, ".codex/config.toml"), trusted_entries.join("\n")) + + previous_path = ENV["PATH"]? + previous_lsp = ENV["AMBER_LSP_BIN"]? + ENV["PATH"] = "#{tools_directory}:#{previous_path}" + ENV["AMBER_LSP_BIN"] = lsp_path + begin + report = AmberCLI::Commands::DoctorCommand.new("doctor", project_root, home_directory).perform + output = report.to_s + + report.exit_code.should eq(0) + output.should contain("[PASS] Claude project hooks are present and current.") + output.should contain("[PASS] Codex project hooks are present and current.") + output.should contain("[PASS] Claude workspace trust is accepted") + output.should contain("[PASS] Claude plugin amber-lsp@amber is enabled without errors.") + output.should contain("[PASS] Codex project hooks are trusted") + output.should contain("[PASS] Amber API lookup index is fresh") + ensure + DoctorCommandSpecHelper.restore_environment("PATH", previous_path) + DoctorCommandSpecHelper.restore_environment("AMBER_LSP_BIN", previous_lsp) + end + codex_settings.should contain(".amber/amber-agent-hook session") + end + end + + it "recognizes plugin list entries that have errors" do + plugin = Array(AmberCLI::Commands::ClaudePluginStatus).from_json( + %([{"id":"amber-lsp@amber","enabled":true,"errors":["manifest invalid"]}]), + ).first + + plugin.has_errors?.should be_true + end + + it "uses the project relative-path marketplace when Claude omits it from plugin list" do + SpecHelper.within_temp_directory do |project_root| + home_directory = File.join(project_root, "home") + tools_directory = File.join(project_root, "tools") + Dir.mkdir_p(home_directory) + codex_settings = DoctorCommandSpecHelper.create_agent_settings(project_root) + AmberCLI::Agent::InstallClaudeAmberLSPPlugin.new.perform + + File.write(File.join(home_directory, ".claude.json"), { + "projects" => { + File.realpath(project_root) => {"hasTrustDialogAccepted" => true}, + }, + }.to_json) + DoctorCommandSpecHelper.write_executable( + File.join(tools_directory, "claude"), + "#!/bin/sh\nprintf '%s\\n' '[]'\n", + ) + DoctorCommandSpecHelper.write_executable(File.join(tools_directory, "codex"), "#!/bin/sh\nexit 0\n") + DoctorCommandSpecHelper.write_executable( + File.join(project_root, ".amber/amber-agent-hook"), + "#!/bin/sh\nprintf '%s\\n' 'Amber agent setup is ready.'\nexit 0\n", + ) + lsp_path = File.join(tools_directory, "amber-lsp") + DoctorCommandSpecHelper.write_executable( + lsp_path, + "#!/bin/sh\n[ \"$1 $2\" = 'lookup --status' ] || exit 2\nprintf '%s\\n' 'Amber API index is fresh'\n", + ) + + codex_settings_path = File.expand_path(File.join(File.realpath(project_root), ".codex/hooks.json")) + trusted_entries = %w[session_start pre_tool_use post_tool_use stop].map do |event| + %([hooks.state."#{codex_settings_path}:#{event}:0:0"]\ntrusted_hash = "sha256:#{"a" * 64}"\n) + end + Dir.mkdir_p(File.join(home_directory, ".codex")) + File.write(File.join(home_directory, ".codex/config.toml"), trusted_entries.join("\n")) + + previous_path = ENV["PATH"]? + previous_lsp = ENV["AMBER_LSP_BIN"]? + ENV["PATH"] = "#{tools_directory}:#{previous_path}" + ENV["AMBER_LSP_BIN"] = lsp_path + begin + report = AmberCLI::Commands::DoctorCommand.new("doctor", project_root, home_directory).perform + output = report.to_s + + report.exit_code.should eq(0) + output.should contain("[PASS] Claude project plugin is declared in its relative-path marketplace; Claude Code omits it from") + ensure + DoctorCommandSpecHelper.restore_environment("PATH", previous_path) + DoctorCommandSpecHelper.restore_environment("AMBER_LSP_BIN", previous_lsp) + end + codex_settings.should contain(".amber/amber-agent-hook session") + end + end +end diff --git a/src/amber_cli.cr b/src/amber_cli.cr index 14ff2cd..9caea74 100644 --- a/src/amber_cli.cr +++ b/src/amber_cli.cr @@ -28,6 +28,7 @@ require "./amber_cli/commands/pipelines" require "./amber_cli/commands/generate" require "./amber_cli/commands/setup_lsp" require "./amber_cli/commands/setup_agent" +require "./amber_cli/commands/doctor" backend = Log::IOBackend.new backend.formatter = Log::Formatter.new do |entry, io| @@ -80,6 +81,7 @@ module AmberCLI pipelines Show application pipelines and plugs setup:lsp (lsp) Set up Amber LSP for Claude Code integration setup:agent (agent) Set up Claude Code and Codex feedback hooks + doctor Check Amber agent setup and trust Options: --version, -v Show version number diff --git a/src/amber_cli/commands/doctor.cr b/src/amber_cli/commands/doctor.cr new file mode 100644 index 0000000..28ecea5 --- /dev/null +++ b/src/amber_cli/commands/doctor.cr @@ -0,0 +1,410 @@ +require "json" +require "../core/base_command" +require "../agent/hook_settings" +require "../agent/find_agent_hook_ignore_rules" + +module AmberCLI::Commands + record DoctorCheck, status : String, message : String do + def ready? : Bool + status != "FAIL" + end + + def to_s(io : IO) : Nil + io << "[" << status << "] " << message + end + end + + class DoctorReport + getter checks : Array(DoctorCheck) + + def initialize(@checks : Array(DoctorCheck)) + end + + def exit_code : Int32 + checks.all?(&.ready?) ? 0 : 1 + end + + def to_s(io : IO) : Nil + checks.each_with_index do |check, index| + io << '\n' if index > 0 + check.to_s(io) + end + end + end + + class ClaudeProjectTrust + include JSON::Serializable + + @[JSON::Field(key: "hasTrustDialogAccepted")] + property has_trust_dialog_accepted : Bool? = nil + + def initialize(@has_trust_dialog_accepted : Bool? = nil) + end + end + + class ClaudeUserConfiguration + include JSON::Serializable + + property projects : Hash(String, ClaudeProjectTrust) = {} of String => ClaudeProjectTrust + + def initialize + end + end + + class ClaudePluginStatus + include JSON::Serializable + + property id : String? = nil + property enabled : Bool? = nil + property errors : Array(String)? = nil + + def initialize + end + + def has_errors? : Bool + list_of_errors = errors + list_of_errors ? !list_of_errors.empty? : false + end + end + + # Checks the project setup and the trust state each agent stores locally. + class DoctorCommand < AmberCLI::Core::BaseCommand + AGENT_HOOK_MODES = %w[session pre post stop] + + def initialize( + command_name : String, + @project_root : String = Dir.current, + @home_directory : String? = ENV["HOME"]?, + @run_external_checks : Bool = true, + ) + super(command_name) + @project_root = File.realpath(@project_root) + end + + def help_description : String + <<-HELP + Check Amber agent setup and local agent trust + + Usage: amber doctor + + Checks amber-lsp, project hooks, Claude Code registration and trust, + Codex hook trust, ignored hook settings, and API lookup index status. + HELP + end + + def setup_command_options + end + + def execute + report = perform + puts report + exit(report.exit_code) unless report.exit_code == 0 + end + + def perform : DoctorReport + checks = [] of DoctorCheck + checks << check_project_hooks(".claude/settings.json", true) + checks << check_project_hooks(".codex/hooks.json", false) + checks.concat(check_agent_readiness) + checks.concat(check_claude_trust) + checks.concat(check_claude_plugin) + checks.concat(check_codex_trust) + checks.concat(check_ignored_settings) + checks << check_api_lookup_index + DoctorReport.new(checks) + end + + private def check_project_hooks(settings_path : String, is_claude_settings : Bool) : DoctorCheck + full_path = File.join(@project_root, settings_path) + platform = is_claude_settings ? "Claude" : "Codex" + unless File.file?(full_path) + return DoctorCheck.new("FAIL", "#{platform} project hooks are missing from #{settings_path}; run `amber setup:agent`.") + end + + settings = AmberCLI::Agent::HookSettings.from_json(File.read(full_path)) + events = settings.hooks + unless events + return DoctorCheck.new("FAIL", "#{platform} project hooks are missing from #{settings_path}; run `amber setup:agent`.") + end + + missing_modes = [] of String + AGENT_HOOK_MODES.each do |mode| + event_groups, matcher = hook_event_groups(events, mode, is_claude_settings) + found = event_groups.any? do |group| + group.matcher == matcher && group.hooks.any? do |handler| + handler.type == "command" && handler.command == ".amber/amber-agent-hook #{mode}" + end + end + missing_modes << mode unless found + end + + if missing_modes.empty? + DoctorCheck.new("PASS", "#{platform} project hooks are present and current.") + else + DoctorCheck.new("FAIL", "#{platform} project hooks are missing or out of date (#{missing_modes.join(", ")}); run `amber setup:agent`.") + end + rescue ex : JSON::ParseException + DoctorCheck.new("FAIL", "Could not parse #{settings_path}: #{ex.message}; run `amber setup:agent`.") + rescue ex : IO::Error + DoctorCheck.new("FAIL", "Could not read #{settings_path}: #{ex.message}; run `amber setup:agent`.") + end + + private def hook_event_groups(events : AmberCLI::Agent::HookEvents, mode : String, is_claude_settings : Bool) : Tuple(Array(AmberCLI::Agent::HookGroup), String?) + case mode + when "session" + {events.session_start, nil} + when "pre" + matcher = is_claude_settings ? AmberCLI::Agent::MergeAgentHooksIntoSettings::CLAUDE_PRE_MATCHER : AmberCLI::Agent::MergeAgentHooksIntoSettings::CODEX_PRE_MATCHER + {events.pre_tool_use, matcher} + when "post" + {events.post_tool_use, AmberCLI::Agent::MergeAgentHooksIntoSettings::POST_MATCHER} + else + {events.stop, nil} + end + end + + private def check_agent_readiness : Array(DoctorCheck) + hook_path = File.join(@project_root, ".amber/amber-agent-hook") + unless @run_external_checks + return [DoctorCheck.new("SKIP", "Amber binary and project coverage check skipped by the doctor test harness.")] + end + unless File::Info.executable?(hook_path) + return [DoctorCheck.new("FAIL", "The agent readiness hook is missing; run `amber setup:agent`.")] + end + + exit_code, output, errors = run_process(hook_path, ["check"], @project_root) + if exit_code == 0 + [DoctorCheck.new("PASS", "Amber agent readiness check passed.")] + else + failures = (output + errors).lines.map(&.strip).reject(&.empty?) + if failures.empty? + failures << "The Amber agent readiness check failed; run `amber setup:agent` and install a supported amber-lsp release." + end + failures.map { |failure| DoctorCheck.new("FAIL", failure) } + end + end + + private def check_claude_trust : Array(DoctorCheck) + unless @run_external_checks + return [DoctorCheck.new("SKIP", "Claude workspace trust check skipped by the doctor test harness.")] + end + executable = Process.find_executable("claude") + unless executable + return [DoctorCheck.new("SKIP", "Claude Code is not installed; workspace trust check skipped.")] + end + unless home = @home_directory + return [DoctorCheck.new("FAIL", "Claude workspace trust could not be checked; set HOME and trust this project in Claude Code.")] + end + + configuration_path = File.join(home, ".claude.json") + unless File.file?(configuration_path) + return [DoctorCheck.new("FAIL", "Claude has no workspace trust record for this project; open it in Claude Code and accept the project trust prompt.")] + end + + configuration = ClaudeUserConfiguration.from_json(File.read(configuration_path)) + project_trust = configuration.projects[@project_root]? + if project_trust.try(&.has_trust_dialog_accepted) == true + [DoctorCheck.new("PASS", "Claude workspace trust is accepted for #{@project_root}.")] + else + [DoctorCheck.new("FAIL", "Claude workspace trust is not accepted for #{@project_root}; open the project in Claude Code and accept the trust prompt.")] + end + rescue ex : JSON::ParseException + [DoctorCheck.new("FAIL", "Could not parse Claude workspace trust: #{ex.message}; open the project in Claude Code and accept the trust prompt.")] + rescue ex : IO::Error + [DoctorCheck.new("FAIL", "Could not read Claude workspace trust: #{ex.message}; open the project in Claude Code and accept the trust prompt.")] + end + + private def check_claude_plugin : Array(DoctorCheck) + unless @run_external_checks + return [DoctorCheck.new("SKIP", "Claude plugin runtime check skipped by the doctor test harness.")] + end + executable = Process.find_executable("claude") + unless executable + return [DoctorCheck.new("SKIP", "Claude Code is not installed; plugin runtime check skipped.")] + end + + exit_code, output, errors = run_process(executable, ["plugin", "list", "--json"], @project_root) + unless exit_code == 0 + detail = (errors + output).strip + detail = "claude plugin list --json failed" if detail.empty? + return [DoctorCheck.new("FAIL", "#{detail}; run `amber setup:agent`, then resolve Claude plugin errors.")] + end + + plugins = Array(ClaudePluginStatus).from_json(output) + plugin = plugins.find { |entry| entry.id == "amber-lsp@amber" } + if plugin && plugin.enabled == true && !plugin.has_errors? + [DoctorCheck.new("PASS", "Claude plugin amber-lsp@amber is enabled without errors.")] + elsif plugin.nil? && relative_amber_lsp_plugin_is_declared? + [DoctorCheck.new("PASS", "Claude project plugin is declared in its relative-path marketplace; Claude Code omits it from claude plugin list --json.")] + else + [DoctorCheck.new("FAIL", "Claude plugin amber-lsp@amber is missing, disabled, or has errors; run `amber setup:agent`, then resolve `claude plugin list --json` errors.")] + end + rescue ex : JSON::ParseException + [DoctorCheck.new("FAIL", "Could not parse `claude plugin list --json`: #{ex.message}; run `amber setup:agent` and resolve Claude plugin errors.")] + end + + private def relative_amber_lsp_plugin_is_declared? : Bool + settings_path = File.join(@project_root, ".claude/settings.json") + return false unless File.file?(settings_path) + + settings = AmberCLI::Agent::HookSettings.from_json(File.read(settings_path)) + marketplaces = settings.extra_known_marketplaces + enabled_plugins = settings.enabled_plugins + return false unless marketplaces && enabled_plugins.try(&.["amber-lsp@amber"]?) == true + + marketplace = marketplaces["amber"]? + return false unless marketplace + + source = marketplace.source.as?(AmberCLI::Agent::ClaudeMarketplaceSourceDetails) + return false unless source.try(&.source) == "directory" + return false unless source.try(&.path) == "./.amber/claude-marketplace" + + [ + ".amber/claude-marketplace/.claude-plugin/marketplace.json", + ".amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json", + ".amber/claude-marketplace/amber-lsp/.lsp.json", + ].all? { |path| File.file?(File.join(@project_root, path)) } + rescue ex : JSON::ParseException + false + rescue ex : IO::Error + false + end + + private def check_codex_trust : Array(DoctorCheck) + unless @run_external_checks + return [DoctorCheck.new("SKIP", "Codex project hook trust check skipped by the doctor test harness.")] + end + executable = Process.find_executable("codex") + unless executable + return [DoctorCheck.new("SKIP", "Codex is not installed; project hook trust check skipped.")] + end + unless home = @home_directory + return [DoctorCheck.new("FAIL", "Codex hook trust could not be checked; set HOME and trust the project hooks in Codex.")] + end + + settings_path = File.join(@project_root, ".codex/hooks.json") + config_path = File.join(home, ".codex/config.toml") + unless File.file?(settings_path) + return [DoctorCheck.new("FAIL", "Codex project hooks are missing; run `amber setup:agent`.")] + end + unless File.file?(config_path) + return [DoctorCheck.new("FAIL", "Codex has no trust records for project hooks; review and trust them in Codex, then run `amber doctor`.")] + end + + settings = AmberCLI::Agent::HookSettings.from_json(File.read(settings_path)) + expected_keys = codex_agent_hook_keys(settings, File.expand_path(settings_path)) + return [DoctorCheck.new("FAIL", "Codex project hooks are missing; run `amber setup:agent`.")] if expected_keys.empty? + + trusted_hashes = codex_trusted_hashes(File.read(config_path)) + missing_keys = expected_keys.reject do |key| + hash = trusted_hashes[key]? + hash && hash.matches?(/\Asha256:[0-9a-fA-F]{64}\z/) + end + if missing_keys.empty? + [DoctorCheck.new("PASS", "Codex project hooks are trusted in #{@project_root}.")] + else + [DoctorCheck.new("FAIL", "Codex project hooks are not all trusted; review and trust #{@project_root}/.codex/hooks.json in Codex, then run `amber doctor`.")] + end + rescue ex : JSON::ParseException + [DoctorCheck.new("FAIL", "Could not parse Codex project hooks: #{ex.message}; review and trust the project hooks in Codex.")] + rescue ex : IO::Error + [DoctorCheck.new("FAIL", "Could not read Codex hook trust: #{ex.message}; review and trust the project hooks in Codex.")] + end + + private def codex_agent_hook_keys(settings : AmberCLI::Agent::HookSettings, settings_path : String) : Array(String) + events = settings.hooks + return [] of String unless events + + keys = [] of String + [ + {"session_start", events.session_start}, + {"pre_tool_use", events.pre_tool_use}, + {"post_tool_use", events.post_tool_use}, + {"stop", events.stop}, + ].each do |event_name, groups| + groups.each_with_index do |group, group_index| + group.hooks.each_with_index do |handler, handler_index| + next unless handler.type == "command" + next unless handler.command.try(&.starts_with?(".amber/amber-agent-hook ")) + + keys << "#{settings_path}:#{event_name}:#{group_index}:#{handler_index}" + end + end + end + keys + end + + private def codex_trusted_hashes(configuration : String) : Hash(String, String) + trusted_hashes = {} of String => String + current_section : String? = nil + configuration.each_line do |line| + if section = line.match(/^\[hooks\.state\."(.*)"\]\s*(?:#.*)?$/) + current_section = section[1] + elsif line.starts_with?('[') + current_section = nil + elsif section_name = current_section + if trusted_hash = line.match(/^\s*trusted_hash\s*=\s*"([^"]*)"/) + trusted_hashes[section_name] = trusted_hash[1] + end + end + end + trusted_hashes + end + + private def check_ignored_settings : Array(DoctorCheck) + ignored_settings = AmberCLI::Agent::FindAgentHookIgnoreRules.new(@project_root).perform + return [DoctorCheck.new("PASS", "Git includes both agent hook settings files in worktrees.")] if ignored_settings.empty? + + ignored_settings.map do |ignored_setting| + DoctorCheck.new( + "FAIL", + "Git ignore rule #{ignored_setting.matching_line} matches #{ignored_setting.path}; remove or narrow that rule so agent worktrees include the hooks.", + ) + end + end + + private def check_api_lookup_index : DoctorCheck + unless @run_external_checks + return DoctorCheck.new("SKIP", "API lookup status check skipped by the doctor test harness.") + end + + binary = resolve_amber_lsp + return DoctorCheck.new("FAIL", "lookup not available in this build") unless binary + + exit_code, output, errors = run_process(binary, ["lookup", "--status"], @project_root) + combined_output = (output + errors).strip + if combined_output.empty? || combined_output.matches?(/(?i)(unknown (command|option)|unrecognized|no such command|lookup not available)/) + return DoctorCheck.new("FAIL", "lookup not available in this build") + end + if exit_code == 0 && combined_output.matches?(/(?i)(fresh|current|up.to.date)/) && !combined_output.matches?(/(?i)(stale|out.of.date|missing)/) + DoctorCheck.new("PASS", "Amber API lookup index is fresh: #{combined_output}") + else + detail = combined_output.empty? ? "amber-lsp lookup --status did not confirm a fresh index" : combined_output + DoctorCheck.new("FAIL", "#{detail}; refresh the API index with `amber-lsp lookup --refresh`.") + end + end + + private def resolve_amber_lsp : String? + if requested_binary = ENV["AMBER_LSP_BIN"]? + if requested_binary.includes?('/') + full_path = File.expand_path(requested_binary, @project_root) + return full_path if File::Info.executable?(full_path) + return nil + end + return Process.find_executable(requested_binary) + end + Process.find_executable("amber-lsp") + end + + private def run_process(command : String, arguments : Array(String), working_directory : String) : Tuple(Int32, String, String) + output = IO::Memory.new + errors = IO::Memory.new + status = Process.run(command, arguments, chdir: working_directory, output: output, error: errors) + {status.exit_code, output.to_s, errors.to_s} + rescue ex : File::NotFoundError + {127, "", ex.message || "executable not found"} + end + end +end + +AmberCLI::Core::CommandRegistry.register("doctor", Array(String).new, AmberCLI::Commands::DoctorCommand) From 0bcef8c4b78092e24e277b8b972fd6ec5862e507 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 21:36:38 -0400 Subject: [PATCH 31/69] feat(amber-cli): set up agent hooks for new projects --- README.md | 41 ++++++++++++++++++++++++------- spec/commands/new_command_spec.cr | 29 ++++++++++++++++++++++ src/amber_cli/commands/new.cr | 30 +++++++++++++++++++++- 3 files changed, 90 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index a674c61..038029c 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,9 @@ amber watch `amber new` installs shards by default and compiles the starter assets. Pass `--no-deps` when an offline or CI workflow needs to run `shards install` later. +It also installs the Claude Code and Codex project hooks and prints an +`amber doctor` summary. Pass `--skip-agent-setup` to generate the app without the +agent files. `amber watch` recompiles assets before the application whenever an ECR template or a file under `app/assets/` changes. Open . @@ -142,8 +145,9 @@ SQL migration to `db/migrations/`, and the resource route to `config/routes.cr`. | `amber assets build` | Supported | Fingerprint `app/assets/` into generated `public/assets/` output | | `amber assets check` | Supported | Verify manifest, bytes, integrity, MIME, and compressed output without changing it | | `amber new APP --type native` | Preview | Not part of the beta platform guarantee | -| `amber setup:lsp` | Available | Configure the bundled diagnostics LSP | -| `amber setup:agent` (`amber agent`) | Available | Install optional Claude Code and Codex feedback hooks | +| `amber setup:lsp` | Available | Register the project-local Claude Amber LSP plugin | +| `amber setup:agent` (`amber agent`) | Available | Install Claude Code and Codex hooks and setup metadata | +| `amber doctor` | Available | Check LSP readiness, hook trust, ignore rules, and API lookup status | Run `amber --help` or `amber COMMAND --help` for command syntax. The detailed [web-app walkthrough](docs/BETA_WEB_APP.md) and @@ -176,19 +180,38 @@ The release archive includes `amber-lsp`. From an Amber project: amber setup:lsp ``` +This writes a tracked directory marketplace under +`.amber/claude-marketplace/` and merges its registration into +`.claude/settings.json`, keeping the project's existing settings. Claude Code +loads the LSP server from that plugin after the user trusts the project folder. + See the [LSP setup guide](https://github.com/amberframework/amber/blob/v2.0.0-beta.5/docs/guides/lsp-setup.md). ## AI agent setup -From an Amber V2 project, run `amber setup:agent` to install Claude Code and -Codex hooks, the project-local `bin/amber-agent-hook`, agent instructions, and -Amber LSP discovery. Existing hook settings and instructions are merged; the -command is safe to run again. The post-edit hook formats and checks only the -changed Crystal file with `amber-lsp --check`; the stop hook asks +`amber new` runs `amber setup:agent` automatically. Run it again after +upgrading Amber CLI to refresh `.amber/agent_setup.json`, +`.amber/amber-agent-hook`, `.claude/settings.json`, `.codex/hooks.json`, and the +Claude plugin files under `.amber/claude-marketplace/`. Existing hook settings +and instructions are merged, so setup is safe to repeat. The generated +SessionStart hook reports whether setup is ready; when setup is incomplete, it +asks the agent to stop and have the user run `amber setup:agent` before editing +Crystal files. The pre-tool hook blocks Crystal writes until the `amber-lsp` +binary meets the recorded minimum version, its checksum matches, and the LSP +covers the project's main entry file. Install the binary with its adjacent +`.sha256` file or the release `checksums.txt` under `share/amber_cli/`. + +Run `amber doctor` to check hook files, Claude workspace trust and plugin +status, Codex project hook trust, Git ignore rules that would hide hooks from +worktrees, and API lookup index freshness. Doctor reads the agent trust records +without changing them. Claude and Codex may need the project and its hooks +trusted before their project-level hooks can run. + +The post-edit hook formats and checks only the changed Crystal file with +`amber-lsp --check`; the stop hook asks `crystal-alpha watch build` for compiler feedback, or checks the target in `shard.yml` with `--no-codegen` when no watcher is running. Install -`crystal-alpha` and `amber-lsp` for the full loop. Codex may ask you to trust -new project hooks before it runs them. +`crystal-alpha` and checksum-verified `amber-lsp` for the full loop. ## Contributing diff --git a/spec/commands/new_command_spec.cr b/spec/commands/new_command_spec.cr index 50da65f..b64f11f 100644 --- a/spec/commands/new_command_spec.cr +++ b/spec/commands/new_command_spec.cr @@ -94,6 +94,18 @@ describe AmberCLI::Commands::NewCommand do command.app_type.should eq("native") command.no_deps.should be_true end + + it "accepts --skip-agent-setup" do + command = AmberCLI::Commands::NewCommand.new("new") + args = ["my_app", "--skip-agent-setup"] + + command.option_parser.unknown_args do |unknown_args, _| + command.remaining_arguments.concat(unknown_args) + end + command.option_parser.parse(args) + + command.skip_agent_setup.should be_true + end end describe "#execute" do @@ -106,6 +118,10 @@ describe AmberCLI::Commands::NewCommand do File.exists?(File.join(destination, "src/beta_smoke.cr")).should be_true Dir.exists?(File.join(destination, "bin")).should be_true + File.file?(File.join(destination, ".amber/agent_setup.json")).should be_true + File.file?(File.join(destination, ".amber/amber-agent-hook")).should be_true + File.file?(File.join(destination, ".claude/settings.json")).should be_true + File.file?(File.join(destination, ".codex/hooks.json")).should be_true shard = File.read(File.join(destination, "shard.yml")) shard.should contain("github: amberframework/amber") @@ -225,5 +241,18 @@ describe AmberCLI::Commands::NewCommand do readme.should contain("public/assets/manifest.json") end end + + it "leaves agent setup for a later command when --skip-agent-setup is passed" do + SpecHelper.within_temp_directory do |temp_dir| + destination = File.join(temp_dir, "skip_agent_app") + command = AmberCLI::Commands::NewCommand.new("new") + + command.parse_and_execute([destination, "--type=web", "--no-deps", "--skip-agent-setup"]) + + File.exists?(File.join(destination, "src/skip_agent_app.cr")).should be_true + File.file?(File.join(destination, ".amber/agent_setup.json")).should be_false + File.file?(File.join(destination, ".codex/hooks.json")).should be_false + end + end end end diff --git a/src/amber_cli/commands/new.cr b/src/amber_cli/commands/new.cr index 04697c6..8078167 100644 --- a/src/amber_cli/commands/new.cr +++ b/src/amber_cli/commands/new.cr @@ -1,6 +1,8 @@ require "../core/base_command" require "../generators/native_app" require "../static_assets" +require "./setup_agent" +require "./doctor" # The `new` command creates a new Amber V2 application with a complete directory # structure, configuration files, and a working home page. @@ -15,6 +17,7 @@ require "../static_assets" # - `-t, --template` - Template language (ECR is the only V2 engine) # - `--type` - Application type: web (default) or native (cross-platform desktop/mobile) # - `--no-deps` - Skip dependency installation +# - `--skip-agent-setup` - Skip Claude Code and Codex hooks # # ## Examples # ``` @@ -41,6 +44,7 @@ module AmberCLI::Commands getter app_type : String = "web" getter assume_yes : Bool = false getter no_deps : Bool = false + getter skip_agent_setup : Bool = false getter name : String = "" # :nodoc: @@ -90,6 +94,11 @@ module AmberCLI::Commands @no_deps = true end + option_parser.on("--skip-agent-setup", "Skip Claude Code and Codex agent setup") do + @parsed_options["skip_agent_setup"] = true + @skip_agent_setup = true + end + option_parser.separator "" option_parser.separator "Usage: amber new [NAME] [options]" option_parser.separator "" @@ -167,6 +176,7 @@ module AmberCLI::Commands info "Test suite:" info " ./mobile/run_all_tests.sh # L1 + L2 tests" info " ./mobile/run_all_tests.sh --e2e # Full E2E tests" + setup_project_agent_loop(full_path_name) end private def execute_web(full_path_name : String, project_name : String) @@ -197,8 +207,26 @@ module AmberCLI::Commands info " amber generate scaffold Pet name:string:required species:string:required" info " amber database migrate" info " amber watch" - info " Tip: run amber setup:agent to install Claude Code and Codex hooks." info " # Choose -d pg or -d mysql when you need a server database." + setup_project_agent_loop(full_path_name) + end + + private def setup_project_agent_loop(project_path : String) : Nil + previous_directory = Dir.current + Dir.cd(project_path) + begin + if skip_agent_setup + info "Skipped Claude Code and Codex setup (--skip-agent-setup)." + else + AmberCLI::Commands::SetupAgentCommand.new("setup:agent").execute + end + + info "" + info "Amber doctor summary:" + puts AmberCLI::Commands::DoctorCommand.new("doctor").perform + ensure + Dir.cd(previous_directory) + end end private def install_dependencies(path : String) From 69faedff943ca194475bd98e2d2eea14d44c58b8 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 22:33:32 -0400 Subject: [PATCH 32/69] fix(amber-cli): anchor agent hook commands to project root --- spec/commands/agent_hook_spec.cr | 46 ++++++++++++++- spec/commands/doctor_spec.cr | 41 ++++++++++++- spec/commands/setup_agent_spec.cr | 19 ++++-- src/amber_cli/agent/hook_settings.cr | 44 +++++++++----- src/amber_cli/commands/doctor.cr | 14 ++++- src/amber_cli/commands/setup_agent.cr | 2 +- .../templates/agent/amber-agent-hook | 59 ++++++++++++++++++- 7 files changed, 198 insertions(+), 27 deletions(-) diff --git a/spec/commands/agent_hook_spec.cr b/spec/commands/agent_hook_spec.cr index c760495..a0336a6 100644 --- a/spec/commands/agent_hook_spec.cr +++ b/spec/commands/agent_hook_spec.cr @@ -16,7 +16,7 @@ module AgentHookSpecHelper File.write(".amber/agent_setup.json", AmberCLI::Agent::AgentSetupManifest.new( AmberCli::VERSION, "1.0.0", - "3", + "4", ).to_pretty_json + "\n") File.write(".amber/claude-marketplace/.claude-plugin/marketplace.json", <<-JSON) { @@ -50,6 +50,10 @@ module AgentHookSpecHelper "enabledPlugins": {"amber-lsp@amber": true} } JSON + claude_settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new(File.read(".claude/settings.json"), true).perform + File.write(".claude/settings.json", claude_settings) + codex_settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", false).perform + File.write(".codex/hooks.json", codex_settings) script = AmberCLI::Commands::SetupAgentCommand::AGENT_HOOK_SCRIPT.gsub("__AMBER_MAIN__", "src/app.cr") hook_path = File.join(project_root, ".amber/amber-agent-hook") @@ -94,6 +98,32 @@ module AgentHookSpecHelper File.write("#{binary_path}.sha256", "#{checksum} #{binary_path}\n") end + def self.write_legacy_agent_hooks : Nil + events = AmberCLI::Agent::HookEvents.new + events.session_start = [legacy_hook_group("session", nil)] + events.pre_tool_use = [legacy_hook_group("pre", AmberCLI::Agent::MergeAgentHooksIntoSettings::CLAUDE_PRE_MATCHER)] + events.post_tool_use = [legacy_hook_group("post", AmberCLI::Agent::MergeAgentHooksIntoSettings::POST_MATCHER)] + events.stop = [legacy_hook_group("stop", nil)] + + claude = AmberCLI::Agent::HookSettings.from_json(File.read(".claude/settings.json")) + claude.hooks = events + File.write(".claude/settings.json", claude.to_pretty_json + "\n") + + codex_events = AmberCLI::Agent::HookEvents.new + codex_events.session_start = [legacy_hook_group("session", nil)] + codex_events.pre_tool_use = [legacy_hook_group("pre", AmberCLI::Agent::MergeAgentHooksIntoSettings::CODEX_PRE_MATCHER)] + codex_events.post_tool_use = [legacy_hook_group("post", AmberCLI::Agent::MergeAgentHooksIntoSettings::POST_MATCHER)] + codex_events.stop = [legacy_hook_group("stop", nil)] + codex = AmberCLI::Agent::HookSettings.new + codex.hooks = codex_events + File.write(".codex/hooks.json", codex.to_pretty_json + "\n") + end + + private def self.legacy_hook_group(mode : String, matcher : String?) : AmberCLI::Agent::HookGroup + hook = AmberCLI::Agent::HookHandler.new("command", ".amber/amber-agent-hook #{mode}") + AmberCLI::Agent::HookGroup.new(matcher, [hook]) + end + def self.run_hook(fixture : AgentHookFixture, mode : String, payload : String = "{}", environment_overrides : Hash(String, String) = {} of String => String) : Tuple(Int32, String, String) output = IO::Memory.new errors = IO::Memory.new @@ -176,6 +206,18 @@ describe "amber-agent-hook readiness and preflight" do end end + it "reports cwd-relative generated hook commands as outdated during readiness" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project(project_root) + AgentHookSpecHelper.write_legacy_agent_hooks + + check = AgentHookSpecHelper.run_hook(fixture, "check") + + check[0].should eq(2) + check[1].should contain("The generated agent hook commands are out of date; run `amber setup:agent`.") + end + end + it "finds amber-lsp on PATH when AMBER_LSP_BIN is unset" do SpecHelper.within_temp_directory do |project_root| fixture = AgentHookSpecHelper.create_ready_project(project_root) @@ -297,7 +339,7 @@ describe "amber-agent-hook readiness and preflight" do first_check = AgentHookSpecHelper.run_hook(fixture, "check") first_check[0].should eq(0) first_calls = File.read(fixture.lsp_log).lines.size - setup_manifest = AmberCLI::Agent::AgentSetupManifest.new(AmberCli::VERSION, "1.0.0", "3") + setup_manifest = AmberCLI::Agent::AgentSetupManifest.new(AmberCli::VERSION, "1.0.0", "4") File.write(".amber/agent_setup.json", setup_manifest.to_json + "\n ") second_check = AgentHookSpecHelper.run_hook(fixture, "check") diff --git a/spec/commands/doctor_spec.cr b/spec/commands/doctor_spec.cr index c8df82b..7e48c4c 100644 --- a/spec/commands/doctor_spec.cr +++ b/spec/commands/doctor_spec.cr @@ -13,6 +13,27 @@ module DoctorCommandSpecHelper codex_settings end + def self.create_legacy_agent_settings(project_root : String) : Nil + [ + {".claude/settings.json", AmberCLI::Agent::MergeAgentHooksIntoSettings::CLAUDE_PRE_MATCHER}, + {".codex/hooks.json", AmberCLI::Agent::MergeAgentHooksIntoSettings::CODEX_PRE_MATCHER}, + ].each do |settings_path, pre_matcher| + events = AmberCLI::Agent::HookEvents.new + events.session_start = [legacy_hook_group("session", nil)] + events.pre_tool_use = [legacy_hook_group("pre", pre_matcher)] + events.post_tool_use = [legacy_hook_group("post", AmberCLI::Agent::MergeAgentHooksIntoSettings::POST_MATCHER)] + events.stop = [legacy_hook_group("stop", nil)] + settings = AmberCLI::Agent::HookSettings.new + settings.hooks = events + File.write(File.join(project_root, settings_path), settings.to_pretty_json + "\n") + end + end + + private def self.legacy_hook_group(mode : String, matcher : String?) : AmberCLI::Agent::HookGroup + hook = AmberCLI::Agent::HookHandler.new("command", ".amber/amber-agent-hook #{mode}") + AmberCLI::Agent::HookGroup.new(matcher, [hook]) + end + def self.write_executable(path : String, content : String) : Nil Dir.mkdir_p(File.dirname(path)) File.write(path, content) @@ -44,6 +65,22 @@ describe "amber doctor" do end end + it "reports cwd-relative generated commands as outdated for Claude and Codex" do + SpecHelper.within_temp_directory do |project_root| + Dir.mkdir_p(File.join(project_root, ".claude")) + Dir.mkdir_p(File.join(project_root, ".codex")) + DoctorCommandSpecHelper.create_legacy_agent_settings(project_root) + + report = AmberCLI::Commands::DoctorCommand.new("doctor", project_root, nil, false).perform + output = report.to_s + + report.exit_code.should eq(1) + output.should contain("[FAIL] Claude project hooks are missing or out of date") + output.should contain("[FAIL] Codex project hooks are missing or out of date") + output.should contain("run `amber setup:agent`.") + end + end + it "reports a ready project with exit status 0 when hooks, trust, plugins, and the API index pass" do SpecHelper.within_temp_directory do |project_root| home_directory = File.join(project_root, "home") @@ -97,7 +134,7 @@ describe "amber doctor" do DoctorCommandSpecHelper.restore_environment("PATH", previous_path) DoctorCommandSpecHelper.restore_environment("AMBER_LSP_BIN", previous_lsp) end - codex_settings.should contain(".amber/amber-agent-hook session") + codex_settings.should contain("git rev-parse --show-toplevel") end end @@ -158,7 +195,7 @@ describe "amber doctor" do DoctorCommandSpecHelper.restore_environment("PATH", previous_path) DoctorCommandSpecHelper.restore_environment("AMBER_LSP_BIN", previous_lsp) end - codex_settings.should contain(".amber/amber-agent-hook session") + codex_settings.should contain("git rev-parse --show-toplevel") end end end diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr index b8139a7..25aeb0b 100644 --- a/spec/commands/setup_agent_spec.cr +++ b/spec/commands/setup_agent_spec.cr @@ -58,12 +58,23 @@ describe "amber setup:agent" do first.should contain(%("Edit|Write|Bash")) end + it "anchors every generated command to the project root for its agent" do + claude = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", true).perform + codex = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", false).perform + + %w[session pre post stop].each do |mode| + claude.should contain("\\\"$CLAUDE_PROJECT_DIR\\\"/.amber/amber-agent-hook #{mode}") + codex.should contain("sh -c 'root=$(git rev-parse --show-toplevel 2>/dev/null) && exec \\\"$root/.amber/amber-agent-hook\\\" #{mode}'") + end + end + it "uses Codex tool names in its pre-edit matcher" do settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", false).perform settings.should contain(%("apply_patch|Bash")) - settings.should contain(".amber/amber-agent-hook session") - settings.should contain(".amber/amber-agent-hook pre") + settings.should contain("git rev-parse --show-toplevel") + settings.should contain(".amber/amber-agent-hook\\\" session") + settings.should contain(".amber/amber-agent-hook\\\" pre") end it "installs the loop into a project without changing existing instructions twice" do @@ -91,7 +102,7 @@ describe "amber setup:agent" do first_claude.should contain(%("allow": [)) first_claude.should contain(".amber/amber-agent-hook session") first_codex.should contain("keep this") - first_codex.should contain(".amber/amber-agent-hook session") + first_codex.should contain("git rev-parse --show-toplevel") first_codex.should contain(%("apply_patch|Bash")) File.read("CLAUDE.md").should eq(first_instructions) File.read("AGENTS.md").scan(/amber-agent-loop:start/).size.should eq(1) @@ -103,7 +114,7 @@ describe "amber setup:agent" do setup_manifest = AmberCLI::Agent::AgentSetupManifest.from_json(File.read(".amber/agent_setup.json")) setup_manifest.amber_cli_version.should eq(AmberCli::VERSION) setup_manifest.minimum_amber_lsp_version.should eq("1.0.0") - setup_manifest.generated_hook_version.should eq("3") + setup_manifest.generated_hook_version.should eq("4") File.file?(".amber/claude-marketplace/amber-lsp/.lsp.json").should be_true File.file?(".lsp.json").should be_false File.info(".amber/amber-agent-hook").permissions.to_i.&(0o111).should_not eq(0) diff --git a/src/amber_cli/agent/hook_settings.cr b/src/amber_cli/agent/hook_settings.cr index 577f86e..09dc135 100644 --- a/src/amber_cli/agent/hook_settings.cr +++ b/src/amber_cli/agent/hook_settings.cr @@ -82,9 +82,19 @@ module AmberCLI::Agent # When an agent loop is installed, retain unrelated hook groups and settings. class MergeAgentHooksIntoSettings - CLAUDE_PRE_MATCHER = "Edit|Write|Bash" - CODEX_PRE_MATCHER = "apply_patch|Bash" - POST_MATCHER = "Edit|Write|MultiEdit|NotebookEdit" + CLAUDE_PRE_MATCHER = "Edit|Write|Bash" + CODEX_PRE_MATCHER = "apply_patch|Bash" + POST_MATCHER = "Edit|Write|MultiEdit|NotebookEdit" + CLAUDE_COMMAND_PREFIX = %("$CLAUDE_PROJECT_DIR"/.amber/amber-agent-hook ) + CODEX_COMMAND_PREFIX = %(sh -c 'root=$(git rev-parse --show-toplevel 2>/dev/null) && exec "$root/.amber/amber-agent-hook" ) + + def self.generated_command_for(mode : String, is_claude_settings : Bool) : String + if is_claude_settings + "#{CLAUDE_COMMAND_PREFIX}#{mode}" + else + "#{CODEX_COMMAND_PREFIX}#{mode}'" + end + end def initialize(@existing_json : String, @is_claude_settings : Bool = true) end @@ -93,21 +103,27 @@ module AmberCLI::Agent settings = @existing_json.empty? ? HookSettings.new : HookSettings.from_json(@existing_json) hooks = settings.hooks || HookEvents.new pre_matcher = @is_claude_settings ? CLAUDE_PRE_MATCHER : CODEX_PRE_MATCHER - remove_stale_pre_hook(hooks.pre_tool_use, pre_matcher, ".amber/amber-agent-hook pre") - add_hook(hooks.session_start, nil, ".amber/amber-agent-hook session") - add_hook(hooks.pre_tool_use, pre_matcher, ".amber/amber-agent-hook pre") - add_hook(hooks.post_tool_use, POST_MATCHER, ".amber/amber-agent-hook post") - add_hook(hooks.stop, nil, ".amber/amber-agent-hook stop") + remove_stale_generated_hooks(hooks) + add_hook(hooks.session_start, nil, self.class.generated_command_for("session", @is_claude_settings)) + add_hook(hooks.pre_tool_use, pre_matcher, self.class.generated_command_for("pre", @is_claude_settings)) + add_hook(hooks.post_tool_use, POST_MATCHER, self.class.generated_command_for("post", @is_claude_settings)) + add_hook(hooks.stop, nil, self.class.generated_command_for("stop", @is_claude_settings)) settings.hooks = hooks settings.to_pretty_json + "\n" end - private def remove_stale_pre_hook(groups : Array(HookGroup), matcher : String, command : String) : Nil - groups.each do |group| - next if group.matcher == matcher - - group.hooks = group.hooks.reject do |handler| - handler.type == "command" && handler.command == command + private def remove_stale_generated_hooks(events : HookEvents) : Nil + { + {"session", events.session_start}, + {"pre", events.pre_tool_use}, + {"post", events.post_tool_use}, + {"stop", events.stop}, + }.each do |mode, groups| + legacy_command = ".amber/amber-agent-hook #{mode}" + groups.each do |group| + group.hooks = group.hooks.reject do |handler| + handler.type == "command" && handler.command == legacy_command + end end end end diff --git a/src/amber_cli/commands/doctor.cr b/src/amber_cli/commands/doctor.cr index 28ecea5..b8cffcd 100644 --- a/src/amber_cli/commands/doctor.cr +++ b/src/amber_cli/commands/doctor.cr @@ -130,12 +130,19 @@ module AmberCLI::Commands missing_modes = [] of String AGENT_HOOK_MODES.each do |mode| event_groups, matcher = hook_event_groups(events, mode, is_claude_settings) + expected_command = AmberCLI::Agent::MergeAgentHooksIntoSettings.generated_command_for(mode, is_claude_settings) + legacy_command = ".amber/amber-agent-hook #{mode}" found = event_groups.any? do |group| group.matcher == matcher && group.hooks.any? do |handler| - handler.type == "command" && handler.command == ".amber/amber-agent-hook #{mode}" + handler.type == "command" && handler.command == expected_command end end - missing_modes << mode unless found + has_legacy_command = event_groups.any? do |group| + group.hooks.any? do |handler| + handler.type == "command" && handler.command == legacy_command + end + end + missing_modes << mode unless found && !has_legacy_command end if missing_modes.empty? @@ -316,6 +323,7 @@ module AmberCLI::Commands return [] of String unless events keys = [] of String + hook_command_prefix = AmberCLI::Agent::MergeAgentHooksIntoSettings::CODEX_COMMAND_PREFIX [ {"session_start", events.session_start}, {"pre_tool_use", events.pre_tool_use}, @@ -325,7 +333,7 @@ module AmberCLI::Commands groups.each_with_index do |group, group_index| group.hooks.each_with_index do |handler, handler_index| next unless handler.type == "command" - next unless handler.command.try(&.starts_with?(".amber/amber-agent-hook ")) + next unless handler.command.try(&.starts_with?(hook_command_prefix)) keys << "#{settings_path}:#{event_name}:#{group_index}:#{handler_index}" end diff --git a/src/amber_cli/commands/setup_agent.cr b/src/amber_cli/commands/setup_agent.cr index 6cce7cd..f317ba3 100644 --- a/src/amber_cli/commands/setup_agent.cr +++ b/src/amber_cli/commands/setup_agent.cr @@ -29,7 +29,7 @@ module AmberCLI::Commands class SetupAgentCommand < AmberCLI::Core::BaseCommand AGENT_HOOK_SCRIPT = {{ read_file("#{__DIR__}/../templates/agent/amber-agent-hook") }} MINIMUM_AMBER_LSP_VERSION = AmberCLI::Agent::MINIMUM_AMBER_LSP_VERSION - GENERATED_HOOK_VERSION = "3" + GENERATED_HOOK_VERSION = "4" DOCUMENT_START = "" DOCUMENT_END = "" diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook index 066ad41..913b16f 100644 --- a/src/amber_cli/templates/agent/amber-agent-hook +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -4,7 +4,7 @@ set -u project_root=$(CDPATH='' cd "$(dirname "$0")/.." && pwd -P) || exit 2 cd "$project_root" || exit 2 -HOOK_VERSION=3 +HOOK_VERSION=4 DEFAULT_MINIMUM_AMBER_LSP_VERSION=1.0.0 MAIN_FILE='__AMBER_MAIN__' @@ -164,6 +164,51 @@ json_string_value() { ' "$json_file" } +generated_hook_commands_are_current() { + awk -v platform="$1" ' + function string_value( character, escaped, result) { + result = ""; escaped = 0 + while (position <= length(document)) { + character = substr(document, position++, 1) + if (escaped) { + if (character == "n") result = result "\n" + else if (character == "r") result = result "\r" + else if (character == "t") result = result "\t" + else result = result character + escaped = 0; continue + } + if (character == "\\") { escaped = 1; continue } + if (character == "\"") return result + result = result character + } + return "" + } + function expected_command(mode) { + if (platform == "claude") return "\"$CLAUDE_PROJECT_DIR\"/.amber/amber-agent-hook " mode + return "sh -c '\''root=$(git rev-parse --show-toplevel 2>/dev/null) && exec \"$root/.amber/amber-agent-hook\" " mode "'\''" + } + { document = document $0 "\n" } + END { + position = 1 + while (position <= length(document)) { + if (substr(document, position++, 1) != "\"") continue + key = string_value() + while (substr(document, position, 1) ~ /[[:space:]]/) position++ + if (substr(document, position++, 1) != ":") continue + while (substr(document, position, 1) ~ /[[:space:]]/) position++ + if (substr(document, position++, 1) != "\"") continue + value = string_value() + if (key == "command") commands[value] = 1 + } + split("session pre post stop", modes, " ") + for (mode_index in modes) { + mode = modes[mode_index] + if (!commands[expected_command(mode)] || commands[".amber/amber-agent-hook " mode]) exit 1 + } + } + ' "$2" +} + add_check_failure() { if [ -n "$CHECK_FAILURES" ]; then CHECK_FAILURES="${CHECK_FAILURES} @@ -292,11 +337,17 @@ setup_cache_key() { .amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json \ .amber/claude-marketplace/amber-lsp/.lsp.json \ .claude/settings.json \ + .codex/hooks.json \ "$MAIN_FILE" \ shard.yml do if [ -f "$dependent_file" ]; then dependent_mtime=$(file_mtime "$dependent_file" 2>/dev/null || printf unknown) + case "$dependent_file" in + .claude/settings.json|.codex/hooks.json) + dependent_mtime="$dependent_mtime:$(cksum < "$dependent_file" 2>/dev/null || printf unknown)" + ;; + esac dependent_mtimes="${dependent_mtimes}${dependent_mtimes:+,}${dependent_file}:${dependent_mtime}" else dependent_mtimes="${dependent_mtimes}${dependent_mtimes:+,}${dependent_file}:missing" @@ -350,6 +401,12 @@ ${cached_failure}" add_check_failure "The project agent setup metadata is missing; run \`amber setup:agent\`." fi + if [ ! -f .claude/settings.json ] || [ ! -f .codex/hooks.json ] || \ + ! generated_hook_commands_are_current claude .claude/settings.json || \ + ! generated_hook_commands_are_current codex .codex/hooks.json; then + add_check_failure "The generated agent hook commands are out of date; run \`amber setup:agent\`." + fi + binary_is_verified=0 if [ -z "$lsp" ] || [ ! -x "$lsp" ]; then requested_binary=${AMBER_LSP_BIN:-amber-lsp} From 5536df4026b4ac86808c037d1edce9cfb2a2a171 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 22:14:30 -0400 Subject: [PATCH 33/69] fix(lsp): format class and extended method lookups --- spec/amber_lsp/lookup/lookup_command_spec.cr | 24 +++++++++++ .../lookup/lsp_lookup_controller_spec.cr | 43 ++++++++++++++++++- .../lookup/resolve_api_query_spec.cr | 2 + src/amber_lsp/controller.cr | 6 +-- src/amber_lsp/lookup/index_models.cr | 41 ++++++++++++++++++ src/amber_lsp/lookup/lookup_command.cr | 4 +- src/amber_lsp/lookup/resolve_api_query.cr | 2 +- 7 files changed, 113 insertions(+), 9 deletions(-) diff --git a/spec/amber_lsp/lookup/lookup_command_spec.cr b/spec/amber_lsp/lookup/lookup_command_spec.cr index 98600e4..0795a2c 100644 --- a/spec/amber_lsp/lookup/lookup_command_spec.cr +++ b/spec/amber_lsp/lookup/lookup_command_spec.cr @@ -62,6 +62,20 @@ describe AmberLSP::Lookup::RunLookupCommand do end end + it "formats class methods, instance methods, and extended class methods by method kind" do + with_lookup_cli_project do |project| + cache_root = lookup_cli_cache_root + class_result = run_lookup_cli(["Dir.mkdir_p", "--root", project], cache_root) + instance_result = run_lookup_cli(["CLIProject::User#id", "--root", project], cache_root) + extended_result = run_lookup_cli(["CLIProject::User.find", "--root", project], cache_root) + + class_result[1].should contain("Dir.mkdir_p(path : Path | String, mode : Int32 = 511) : Nil") + class_result[1].should_not contain("Dir#mkdir_p") + instance_result[1].should contain("CLIProject::User#id() : Int64 | Nil") + extended_result[1].should contain("CLIProject::Querying::ClassMethods.find() : String (class method via extend)") + end + end + it "uses exit code 5 for a compiler-verified absent method" do with_lookup_cli_project do |project| result = run_lookup_cli( @@ -150,7 +164,17 @@ private def with_lookup_cli_project(&) YAML File.write(source_path, <<-CRYSTAL) module CLIProject + module Querying + module ClassMethods + def find : String + "found" + end + end + end + class Parent + extend Querying::ClassMethods + def inherited_api end end diff --git a/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr b/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr index ebda936..b528e79 100644 --- a/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr +++ b/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr @@ -38,6 +38,18 @@ describe AmberLSP::Controller do workspace_symbol["name"].as_s.should eq("FixtureAPI::User#id()") workspace_symbol["data"]["freshness"].as_s.should eq("fresh") + extension_workspace_response = controller.handle( + lsp_lookup_request(6, "workspace/symbol", {"query" => "FixtureAPI::Team.where"}), + server, + ) + extension_workspace_json = extension_workspace_response || raise "Expected extended method symbol response" + extension_workspace = AmberLSP::Lookup::LSPWorkspaceSymbolResponse.from_json(extension_workspace_json) + extension_workspace_symbol = extension_workspace.list_of_workspace_symbols.first? || + raise "Expected an extended method workspace symbol" + extension_workspace_symbol.symbol_name.should eq( + "FixtureAPI::QueryMethods.where(field : String) (class method via extend)", + ) + hover_value = hover_json["result"]["contents"]["value"].as_s hover_value.should contain("FixtureAPI::User#id() : Int64 | Nil") hover_value.should contain("The user's primary key.") @@ -45,6 +57,32 @@ describe AmberLSP::Controller do hover_value.should contain("Freshness: fresh") hover_value.should contain("The ID getter is generated from the persisted table column.") + class_hover_response = controller.handle( + lsp_lookup_request(4, "textDocument/hover", { + "textDocument" => {"uri" => uri}, + "position" => {"line" => 1, "character" => 21}, + }), + server, + ) + class_hover_json = class_hover_response || raise "Expected class method hover response" + class_hover = AmberLSP::Lookup::LSPHoverResponse.from_json(class_hover_json) + class_hover_result = class_hover.result || raise "Expected class method hover contents" + class_hover_value = class_hover_result.contents.markdown_value + class_hover_value.should contain("FixtureAPI::User.count() : Int32") + + extension_hover_response = controller.handle( + lsp_lookup_request(5, "textDocument/hover", { + "textDocument" => {"uri" => uri}, + "position" => {"line" => 2, "character" => 21}, + }), + server, + ) + extension_hover_json = extension_hover_response || raise "Expected extended method hover response" + extension_hover = AmberLSP::Lookup::LSPHoverResponse.from_json(extension_hover_json) + extension_hover_result = extension_hover.result || raise "Expected extended method hover contents" + extension_hover_value = extension_hover_result.contents.markdown_value + extension_hover_value.should contain("FixtureAPI::QueryMethods.where(field : String) : Query (class method via extend)") + definition_json["result"].as_a.first["uri"].as_s.should eq("file://#{File.join(project, "src", "models", "user.cr")}") definition_json["result"].as_a.first["range"]["start"]["line"].as_i.should eq(11) end @@ -120,7 +158,10 @@ private def with_lsp_lookup_project(&) Dir.mkdir_p(source_root) Dir.mkdir_p(api_card_root) File.write(File.join(project, "shard.yml"), "name: fixture_project\nversion: 1.5.0\n") - File.write(File.join(source_root, "user.cr"), "FixtureAPI::User#id\n") + File.write( + File.join(source_root, "user.cr"), + "FixtureAPI::User#id\nFixtureAPI::User.count\nFixtureAPI::Team.where\n", + ) card_fixture = File.join(Dir.current, "spec", "fixtures", "api_lookup", "cards", "project", ".amber-lsp") FileUtils.cp_r(card_fixture, project) diff --git a/spec/amber_lsp/lookup/resolve_api_query_spec.cr b/spec/amber_lsp/lookup/resolve_api_query_spec.cr index 2f9646c..7ae6df9 100644 --- a/spec/amber_lsp/lookup/resolve_api_query_spec.cr +++ b/spec/amber_lsp/lookup/resolve_api_query_spec.cr @@ -29,8 +29,10 @@ describe "AmberLSP::Lookup::ResolveAPIQuery#perform" do direct_result.resolution_kind.should eq("class_method") direct_result.list_of_methods.size.should eq(2) direct_result.list_of_methods.all? { |method| method.owner == "FixtureAPI::QueryMethods" }.should be_true + direct_result.list_of_methods.all? { |method| method.method_kind == "extended_class" }.should be_true ancestor_result.list_of_methods.size.should eq(2) ancestor_result.list_of_methods.all? { |method| method.owner == "FixtureAPI::QueryMethods" }.should be_true + ancestor_result.list_of_methods.all? { |method| method.method_kind == "extended_class" }.should be_true end it "matches generic extension owners by their base name" do diff --git a/src/amber_lsp/controller.cr b/src/amber_lsp/controller.cr index 189d053..ee3acc1 100644 --- a/src/amber_lsp/controller.cr +++ b/src/amber_lsp/controller.cr @@ -320,7 +320,7 @@ module AmberLSP notes, ) list_of_symbols << Lookup::LSPWorkspaceSymbol.new( - "#{entry.owner}##{entry.name}#{entry.args_string}", + entry.lookup_symbol_name, 6, location, entry.owner, @@ -454,10 +454,8 @@ module AmberLSP ) : String lines = [] of String answer.list_of_entries.each do |entry| - argument_string = entry.args_string.starts_with?('(') ? entry.args_string : "(#{entry.args_string})" - return_type = entry.return_type.gsub("::Nil", "Nil") lines << "```crystal" - lines << "#{entry.owner}##{entry.name}#{argument_string} : #{return_type}" + lines << entry.lookup_signature lines << "```" if doc_line = entry.doc_line lines << doc_line diff --git a/src/amber_lsp/lookup/index_models.cr b/src/amber_lsp/lookup/index_models.cr index ede044f..bf5aa29 100644 --- a/src/amber_lsp/lookup/index_models.cr +++ b/src/amber_lsp/lookup/index_models.cr @@ -127,6 +127,47 @@ module AmberLSP::Lookup @macro : Bool, ) end + + def as_extended_class_method : APIIndexMethod + APIIndexMethod.new( + owner, + name, + "extended_class", + args_string, + return_type, + doc_line, + source_path, + source_line, + source_layer, + abstract?, + macro?, + ) + end + + def lookup_reference : String + separator = method_kind == "instance" ? "#" : "." + "#{owner}#{separator}#{name}" + end + + def lookup_label : String + "#{lookup_reference}#{lookup_argument_list}" + end + + def lookup_symbol_name : String + "#{lookup_label}#{extension_annotation}" + end + + def lookup_signature : String + "#{lookup_label} : #{return_type.gsub("::Nil", "Nil")}#{extension_annotation}" + end + + private def lookup_argument_list : String + args_string.starts_with?('(') ? args_string : "(#{args_string})" + end + + private def extension_annotation : String + method_kind == "extended_class" ? " (class method via extend)" : "" + end end # :nodoc: diff --git a/src/amber_lsp/lookup/lookup_command.cr b/src/amber_lsp/lookup/lookup_command.cr index c671fda..69b95be 100644 --- a/src/amber_lsp/lookup/lookup_command.cr +++ b/src/amber_lsp/lookup/lookup_command.cr @@ -222,9 +222,7 @@ module AmberLSP::Lookup end answer.list_of_entries.each do |entry| - argument_string = entry.args_string.starts_with?('(') ? entry.args_string : "(#{entry.args_string})" - return_type = entry.return_type.gsub("::Nil", "Nil") - @stdout.puts("#{entry.owner}##{entry.name}#{argument_string} : #{return_type} — #{entry.source_path}:#{entry.source_line} [#{entry.source_layer}]") + @stdout.puts("#{entry.lookup_signature} — #{entry.source_path}:#{entry.source_line} [#{entry.source_layer}]") @stdout.puts(" #{entry.doc_line}") if entry.doc_line end diff --git a/src/amber_lsp/lookup/resolve_api_query.cr b/src/amber_lsp/lookup/resolve_api_query.cr index e374273..2c24ce0 100644 --- a/src/amber_lsp/lookup/resolve_api_query.cr +++ b/src/amber_lsp/lookup/resolve_api_query.cr @@ -107,7 +107,7 @@ module AmberLSP::Lookup visited_module_names.add(module_name) types_named(module_name).each do |module_type| - methods.concat(methods_named([module_type], method_name, "instance")) + methods.concat(methods_named([module_type], method_name, "instance").map(&.as_extended_class_method)) end end end From ac1b116d7d11bf2dd434b6440546b0ed48af6d6a Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 22:17:43 -0400 Subject: [PATCH 34/69] fix(lsp): fail coverage when rule packs cannot load --- spec/amber_lsp/rule_packs_spec.cr | 39 ++++++++++++++++++- src/amber_lsp/analyze_file_with_coverage.cr | 3 ++ src/amber_lsp/analyzer.cr | 12 +++++- .../load_rule_packs_for_project.cr | 19 +++++++++ 4 files changed, 70 insertions(+), 3 deletions(-) diff --git a/spec/amber_lsp/rule_packs_spec.cr b/spec/amber_lsp/rule_packs_spec.cr index c93efec..42297e1 100644 --- a/spec/amber_lsp/rule_packs_spec.cr +++ b/spec/amber_lsp/rule_packs_spec.cr @@ -1,4 +1,5 @@ require "./spec_helper" +require "../../src/amber_lsp/check_file_for_diagnostics" GRANT_TENANCY_PACK_FIXTURE_PATH = File.join( Dir.current, @@ -112,17 +113,51 @@ describe "AmberLSP Grant tenancy rule pack v2" do end end - it "skips malformed and unreadable pack files" do + it "records malformed, invalid, and unreadable pack files" do with_tempdir do |root| pack_directory = File.join(root, "lib", "grant", ".amber-lsp", "packs") Dir.mkdir_p(pack_directory) File.write(File.join(pack_directory, "malformed.yml"), "pack: [\n") + File.write(File.join(pack_directory, "invalid.yml"), "pack: test\n") Dir.mkdir(File.join(pack_directory, "unreadable.yml")) project_context = AmberLSP::ProjectContext.new(root) - list_of_rule_packs = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs + loader = AmberLSP::LibraryRulePacks::LoadRulePacksForProject.new(project_context) + list_of_rule_packs = loader.load_rule_packs list_of_rule_packs.should be_empty + loader.list_of_load_failures.size.should eq(3) + loader.list_of_load_failures.map(&.pack_path).should contain(File.join(pack_directory, "malformed.yml")) + loader.list_of_load_failures.map(&.pack_path).should contain(File.join(pack_directory, "invalid.yml")) + loader.list_of_load_failures.map(&.pack_path).should contain(File.join(pack_directory, "unreadable.yml")) + end + end + + it "fails coverage and --check when a library rule pack cannot be loaded" do + with_tempdir do |root| + install_tenancy_fixture_app(root, "row_app") + File.write( + File.join(root, "shard.yml"), + "name: malformed_pack_app\nversion: 0.1.0\ndependencies:\n grant:\n github: amberframework/grant\n", + ) + pack_path = File.join(root, "lib", "grant", ".amber-lsp", "packs", "tenancy.yml") + File.write(pack_path, "pack: [\n") + file_path = File.join(root, "src", "controllers", "invoices_controller.cr") + content = "Invoice.unscoped.all\n" + File.write(file_path, content) + + coverage = AmberLSP::AnalyzeFileWithCoverage.new(file_path, content).perform + coverage.should be_a(AmberLSP::Coverage::Failed) + if failed = coverage.as?(AmberLSP::Coverage::Failed) + failed.error.should contain(pack_path) + failed.error.should contain("line") + end + + output = IO::Memory.new + exit_code = AmberLSP::CheckFileForDiagnostics.new(file_path, output).perform + exit_code.should eq(2) + output.to_s.should contain("amber-lsp: failed") + output.to_s.should contain(pack_path) end end diff --git a/src/amber_lsp/analyze_file_with_coverage.cr b/src/amber_lsp/analyze_file_with_coverage.cr index 229b5af..6f5bf6b 100644 --- a/src/amber_lsp/analyze_file_with_coverage.cr +++ b/src/amber_lsp/analyze_file_with_coverage.cr @@ -28,6 +28,9 @@ module AmberLSP analyzer = analyzer_for(current_project_context) @analysis_analyzer = analyzer + if error = analyzer.rule_pack_load_failure + return Coverage::Failed.new(error) + end Coverage::Covered.new(analyzer.analyze(@file_path, current_content)) rescue ex : Exception Coverage::Failed.new("#{ex.class}: #{ex.message}") diff --git a/src/amber_lsp/analyzer.cr b/src/amber_lsp/analyzer.cr index 14f13ce..9f93076 100644 --- a/src/amber_lsp/analyzer.cr +++ b/src/amber_lsp/analyzer.cr @@ -4,19 +4,23 @@ module AmberLSP @project_root : String? @project_context : ProjectContext? @library_rule_pack_analyzer : LibraryRulePacks::AnalyzeProjectFilesWithRulePacks? + @list_of_rule_pack_load_failures : Array(LibraryRulePacks::RulePackLoadFailure) def initialize @configuration = Configuration.new @project_root = nil @project_context = nil @library_rule_pack_analyzer = nil + @list_of_rule_pack_load_failures = [] of LibraryRulePacks::RulePackLoadFailure end def configure(project_context : ProjectContext) : Nil @configuration = Configuration.load(project_context.root_path) @project_root = project_context.root_path @project_context = project_context - list_of_rule_packs = LibraryRulePacks::LoadRulePacksForProject.new(project_context).load_rule_packs + rule_pack_loader = LibraryRulePacks::LoadRulePacksForProject.new(project_context) + list_of_rule_packs = rule_pack_loader.load_rule_packs + @list_of_rule_pack_load_failures = rule_pack_loader.list_of_load_failures @library_rule_pack_analyzer = LibraryRulePacks::AnalyzeProjectFilesWithRulePacks.new( project_context, @configuration, @@ -25,6 +29,12 @@ module AmberLSP register_custom_rules end + def rule_pack_load_failure : String? + return nil if @list_of_rule_pack_load_failures.empty? + + @list_of_rule_pack_load_failures.map(&.formatted_error).join("\n") + end + def has_applicable_library_rule_pack?(file_path : String, content : String) : Bool analyzer = @library_rule_pack_analyzer return false unless analyzer diff --git a/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr b/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr index 5fb79f2..a2c910a 100644 --- a/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr +++ b/src/amber_lsp/library_rule_packs/load_rule_packs_for_project.cr @@ -1,12 +1,29 @@ require "log" module AmberLSP::LibraryRulePacks + # :nodoc: + struct RulePackLoadFailure + getter pack_path : String + getter reason : String + + def initialize(@pack_path : String, @reason : String) + end + + def formatted_error : String + "#{pack_path}: #{reason}" + end + end + class LoadRulePacksForProject + getter list_of_load_failures : Array(RulePackLoadFailure) + def initialize(@project_context : AmberLSP::ProjectContext) + @list_of_load_failures = [] of RulePackLoadFailure end def load_rule_packs : Array(DescribeLibraryRulePack) list_of_packs_by_id = {} of String => DescribeLibraryRulePack + @list_of_load_failures.clear dependency_pack_paths.each do |pack_path| load_rule_pack(pack_path).try do |rule_pack| @@ -38,9 +55,11 @@ module AmberLSP::LibraryRulePacks rule_pack = DescribeLibraryRulePack.from_yaml(read_rule_pack_contents(pack_path)) return rule_pack if rule_pack_is_valid?(rule_pack) + @list_of_load_failures << RulePackLoadFailure.new(pack_path, "invalid rule pack schema") Log.warn { "Ignoring invalid amber-lsp rule pack at #{pack_path}." } nil rescue ex : YAML::ParseException | IO::Error + @list_of_load_failures << RulePackLoadFailure.new(pack_path, ex.message || ex.class.to_s) Log.warn(exception: ex) do "Could not load amber-lsp rule pack at #{pack_path} (#{ex.class}): #{ex.message}" end From 14b4ac10372af4b946244d51eefa5c6c3efa0918 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 22:21:25 -0400 Subject: [PATCH 35/69] fix(lsp): propagate tenancy source parse failures --- spec/amber_lsp/rule_packs_spec.cr | 50 +++++++++++++++++-- .../determine_project_rule_pack_state.cr | 2 +- ...lect_project_grant_tenancy_declarations.cr | 12 +++-- 3 files changed, 56 insertions(+), 8 deletions(-) diff --git a/spec/amber_lsp/rule_packs_spec.cr b/spec/amber_lsp/rule_packs_spec.cr index 42297e1..985672d 100644 --- a/spec/amber_lsp/rule_packs_spec.cr +++ b/spec/amber_lsp/rule_packs_spec.cr @@ -144,6 +144,7 @@ describe "AmberLSP Grant tenancy rule pack v2" do File.write(pack_path, "pack: [\n") file_path = File.join(root, "src", "controllers", "invoices_controller.cr") content = "Invoice.unscoped.all\n" + Dir.mkdir_p(File.dirname(file_path)) File.write(file_path, content) coverage = AmberLSP::AnalyzeFileWithCoverage.new(file_path, content).perform @@ -161,6 +162,35 @@ describe "AmberLSP Grant tenancy rule pack v2" do end end + it "fails coverage when a row-tenant source file has a syntax error" do + with_tempdir do |root| + install_tenancy_fixture_app(root, "row_app") + File.write( + File.join(root, "shard.yml"), + "name: malformed_model_app\nversion: 0.1.0\ndependencies:\n grant:\n github: amberframework/grant\n", + ) + model_file_path = File.join(root, "src", "models", "invoice.cr") + model_content = "class Invoice < Grant::Base\n multitenant :account_id\n column account_id : Int64\n" + File.write(model_file_path, model_content) + controller_file_path = File.join(root, "src", "controllers", "invoices_controller.cr") + controller_content = "Invoice.unscoped.all\n" + Dir.mkdir_p(File.dirname(controller_file_path)) + File.write(controller_file_path, controller_content) + + coverage = AmberLSP::AnalyzeFileWithCoverage.new(controller_file_path, controller_content).perform + coverage.should be_a(AmberLSP::Coverage::Failed) + if failed = coverage.as?(AmberLSP::Coverage::Failed) + failed.error.should contain("#{model_file_path}:4:") + end + + output = IO::Memory.new + exit_code = AmberLSP::CheckFileForDiagnostics.new(controller_file_path, output).perform + exit_code.should eq(2) + output.to_s.should contain("amber-lsp: failed") + output.to_s.should contain("#{model_file_path}:4:") + end + end + it "propagates unexpected pack read errors" do with_tempdir do |root| pack_path = File.join(root, "lib", "grant", ".amber-lsp", "packs", "tenancy.yml") @@ -585,15 +615,27 @@ describe "AmberLSP Grant tenancy rule pack v2" do end end - it "skips malformed Crystal files without crashing or inventing another mode" do + it "fails coverage for malformed project Crystal files" do with_tempdir do |root| install_tenancy_fixture_app(root, "row_app") + File.write( + File.join(root, "shard.yml"), + "name: malformed_source_app\nversion: 0.1.0\ndependencies:\n grant:\n github: amberframework/grant\n", + ) Dir.mkdir_p(File.join(root, "config")) - File.write(File.join(root, "config", "broken.cr"), "Grant::SchemaTenant.with(\"broken\") do\n") + broken_file_path = File.join(root, "config", "broken.cr") + File.write(broken_file_path, "Grant::SchemaTenant.with(\"broken\") do\n") + file_path = File.join(root, "src", "jobs", "clean_job.cr") + content = "Invoice.all\n" + Dir.mkdir_p(File.dirname(file_path)) + File.write(file_path, content) - diagnostics = analyze_tenancy_fixture_source(root, "src/jobs/clean_job.cr", "Invoice.all\n") + coverage = AmberLSP::AnalyzeFileWithCoverage.new(file_path, content).perform - diagnostics.should be_empty + coverage.should be_a(AmberLSP::Coverage::Failed) + if failed = coverage.as?(AmberLSP::Coverage::Failed) + failed.error.should contain(broken_file_path) + end end end end diff --git a/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr b/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr index 0ce69af..816909f 100644 --- a/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr +++ b/src/amber_lsp/library_rule_packs/determine_project_rule_pack_state.cr @@ -142,7 +142,7 @@ module AmberLSP::LibraryRulePacks end private def collect_project_declarations_from(file_path : String, content : String) : Nil - collector = GrantTenancy::CollectProjectGrantTenancyDeclarations.for_source(content) + collector = GrantTenancy::CollectProjectGrantTenancyDeclarations.for_source(content, file_path) @uses_row_tenancy ||= collector.uses_row_tenancy? @uses_schema_tenancy ||= collector.uses_schema_tenancy? @list_of_model_declarations_by_file[file_path] = collector.list_of_model_declarations diff --git a/src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations.cr b/src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations.cr index eb2acb4..6a0a77a 100644 --- a/src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations.cr +++ b/src/amber_lsp/library_rule_packs/grant_tenancy/collect_project_grant_tenancy_declarations.cr @@ -1,6 +1,10 @@ require "compiler/crystal/syntax" module AmberLSP::LibraryRulePacks::GrantTenancy + # :nodoc: + class ProjectSourceParseError < Exception + end + class CollectProjectGrantTenancyDeclarations < Crystal::Visitor getter list_of_model_declarations : Array(GrantTenantModelDeclaration) getter? uses_row_tenancy : Bool @@ -19,12 +23,14 @@ module AmberLSP::LibraryRulePacks::GrantTenancy @uses_schema_tenancy = false end - def self.for_source(content : String) : CollectProjectGrantTenancyDeclarations + def self.for_source(content : String, source_path : String) : CollectProjectGrantTenancyDeclarations collector = new collector.accept(Crystal::Parser.new(content).parse) collector - rescue Crystal::SyntaxException - new + rescue ex : Crystal::SyntaxException + raise ProjectSourceParseError.new( + "Crystal syntax error in #{source_path}:#{ex.line_number}: #{ex.message || "invalid Crystal source"}", + ) end def visit(node : Crystal::ASTNode) : Bool From d56c7650008e63d4a251a81a5c341673a4fdab3a Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 22:24:21 -0400 Subject: [PATCH 36/69] feat(lsp): add compiler error hint command --- spec/amber_lsp/lookup/hint_command_spec.cr | 78 ++++++++++++++++++++++ src/amber_lsp.cr | 3 + src/amber_lsp/lookup/api_cards.cr | 16 +++++ src/amber_lsp/lookup/hint_command.cr | 59 ++++++++++++++++ 4 files changed, 156 insertions(+) create mode 100644 spec/amber_lsp/lookup/hint_command_spec.cr create mode 100644 src/amber_lsp/lookup/hint_command.cr diff --git a/spec/amber_lsp/lookup/hint_command_spec.cr b/spec/amber_lsp/lookup/hint_command_spec.cr new file mode 100644 index 0000000..456f355 --- /dev/null +++ b/spec/amber_lsp/lookup/hint_command_spec.cr @@ -0,0 +1,78 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/hint_command" + +describe AmberLSP::Lookup::RunHintCommand do + it "prints matching project and dependency card hints from compiler output" do + with_hint_cli_project do |project| + input = IO::Memory.new("undefined method 'save'\nundefined method 'lookup'\n") + output = IO::Memory.new + error = IO::Memory.new + exit_code = AmberLSP::Lookup::RunHintCommand.new( + ["--root", project], + input, + output, + error, + ).perform + + exit_code.should eq(0) + output.to_s.lines.should eq([ + "hint: Persist the model with its supported save operation. [fixture_project@1.5.0]", + " right: user.save", + "hint: Use the lookup helper defined on the widget's query module. [example@0.3.5]", + " right: Example::Widget.lookup(\"widget-1\")", + ]) + error.to_s.should be_empty + end + end + + it "exits successfully without output when no card hint matches" do + with_hint_cli_project do |project| + input = IO::Memory.new("undefined method 'unlisted'\n") + output = IO::Memory.new + code = AmberLSP::Lookup::RunHintCommand.new( + ["--root", project], + input, + output, + IO::Memory.new, + ).perform + + code.should eq(0) + output.to_s.should be_empty + end + end + + it "prints usage and exits successfully for --help" do + output = IO::Memory.new + code = AmberLSP::Lookup::RunHintCommand.new( + ["--help"], + IO::Memory.new, + output, + IO::Memory.new, + ).perform + + code.should eq(0) + output.to_s.should contain("Usage: amber-lsp hint [--root DIR]") + output.to_s.should contain("compiler output from stdin") + end +end + +private def with_hint_cli_project(&) + with_tempdir do |project| + fixture_root = File.join(Dir.current, "spec", "fixtures", "api_lookup", "cards") + File.write( + File.join(project, "shard.yml"), + "name: fixture_project\nversion: 1.5.0\ndependencies:\n example:\n github: example/library\n", + ) + File.write( + File.join(project, "shard.lock"), + "shards:\n example:\n version: 0.3.5\n", + ) + Dir.mkdir_p(File.join(project, "lib", "example")) + FileUtils.cp_r(File.join(fixture_root, "project", ".amber-lsp"), project) + FileUtils.cp_r( + File.join(fixture_root, "library", ".amber-lsp"), + File.join(project, "lib", "example"), + ) + yield project + end +end diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index 4eb90a9..f9ff43d 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -46,12 +46,15 @@ require "./amber_lsp/lookup/resolve_api_query" require "./amber_lsp/lookup/answer_api_query" require "./amber_lsp/lookup/verify_api_query" require "./amber_lsp/lookup/lookup_command" +require "./amber_lsp/lookup/hint_command" if ARGV.includes?("--version") STDOUT.puts AmberLSP.version_line exit 0 elsif ARGV.first? == "lookup" exit AmberLSP::Lookup::RunLookupCommand.new(ARGV[1..].to_a).perform +elsif ARGV.first? == "hint" + exit AmberLSP::Lookup::RunHintCommand.new(ARGV[1..].to_a).perform elsif ARGV[0]? == "--check" file_path = ARGV[1]? unless file_path diff --git a/src/amber_lsp/lookup/api_cards.cr b/src/amber_lsp/lookup/api_cards.cr index 729cbae..4bf6d37 100644 --- a/src/amber_lsp/lookup/api_cards.cr +++ b/src/amber_lsp/lookup/api_cards.cr @@ -42,6 +42,16 @@ module AmberLSP::Lookup end end + # :nodoc: + struct APICardErrorHintMatch + getter card_library : String + getter card_version : String + getter error_hint : APICardErrorHint + + def initialize(@card_library : String, @card_version : String, @error_hint : APICardErrorHint) + end + end + struct APICardCollection getter list_of_cards : Array(LoadedAPICard) getter list_of_errors : Array(String) @@ -67,9 +77,15 @@ module AmberLSP::Lookup end def matching_error_hints(error_text : String) : Array(APICardErrorHint) + matching_error_hint_matches(error_text).map(&.error_hint) + end + + def matching_error_hint_matches(error_text : String) : Array(APICardErrorHintMatch) @list_of_cards.flat_map do |loaded_card| loaded_card.card.error_hints.select do |hint| Regex.new(hint.pattern).matches?(error_text) + end.map do |hint| + APICardErrorHintMatch.new(loaded_card.card.library, loaded_card.resolved_version, hint) end end end diff --git a/src/amber_lsp/lookup/hint_command.cr b/src/amber_lsp/lookup/hint_command.cr new file mode 100644 index 0000000..e2112e1 --- /dev/null +++ b/src/amber_lsp/lookup/hint_command.cr @@ -0,0 +1,59 @@ +require "file_utils" + +require "./api_cards" + +module AmberLSP::Lookup + class RunHintCommand + USAGE = "Usage: amber-lsp hint [--root DIR]\nReads Crystal compiler output from stdin and shows matching API card hints." + + def initialize( + @arguments : Array(String), + @input : IO = STDIN, + @stdout : IO = STDOUT, + @stderr : IO = STDERR, + ) + end + + def perform : Int32 + if @arguments.includes?("--help") || @arguments.includes?("-h") + @stdout.puts(USAGE) + return 0 + end + + root_path = project_root_path + error_text = @input.gets_to_end + card_collection = LoadAPICards.new(root_path).perform + print_matching_hints(card_collection.matching_error_hint_matches(error_text)) + 0 + rescue ex : ArgumentError | IO::Error + @stderr.puts("amber-lsp hint failed: #{ex.message || ex.class.to_s}") + 2 + end + + private def project_root_path : String + root_path = Dir.current + argument_index = 0 + + while argument_index < @arguments.size + argument = @arguments[argument_index] + case argument + when "--root" + argument_index += 1 + root_path = @arguments[argument_index]? || raise ArgumentError.new("--root requires a directory") + else + raise ArgumentError.new("unexpected argument #{argument.inspect}") + end + argument_index += 1 + end + + File.expand_path(root_path) + end + + private def print_matching_hints(list_of_matches : Array(APICardErrorHintMatch)) : Nil + list_of_matches.each do |match| + @stdout.puts("hint: #{match.error_hint.hint} [#{match.card_library}@#{match.card_version}]") + @stdout.puts(" right: #{match.error_hint.example}") + end + end + end +end From 080efbf60676cce31454b637c5e3e152b153db14 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 22:38:55 -0400 Subject: [PATCH 37/69] fix(amber-cli): match agent post tool payloads --- spec/commands/agent_hook_spec.cr | 4 +- spec/commands/doctor_spec.cr | 2 +- spec/commands/setup_agent_spec.cr | 51 ++++++- src/amber_cli/agent/hook_settings.cr | 17 ++- src/amber_cli/commands/doctor.cr | 2 +- .../templates/agent/amber-agent-hook | 127 +++++++++--------- 6 files changed, 125 insertions(+), 78 deletions(-) diff --git a/spec/commands/agent_hook_spec.cr b/spec/commands/agent_hook_spec.cr index a0336a6..fc0a1c2 100644 --- a/spec/commands/agent_hook_spec.cr +++ b/spec/commands/agent_hook_spec.cr @@ -102,7 +102,7 @@ module AgentHookSpecHelper events = AmberCLI::Agent::HookEvents.new events.session_start = [legacy_hook_group("session", nil)] events.pre_tool_use = [legacy_hook_group("pre", AmberCLI::Agent::MergeAgentHooksIntoSettings::CLAUDE_PRE_MATCHER)] - events.post_tool_use = [legacy_hook_group("post", AmberCLI::Agent::MergeAgentHooksIntoSettings::POST_MATCHER)] + events.post_tool_use = [legacy_hook_group("post", "Edit|Write|MultiEdit|NotebookEdit")] events.stop = [legacy_hook_group("stop", nil)] claude = AmberCLI::Agent::HookSettings.from_json(File.read(".claude/settings.json")) @@ -112,7 +112,7 @@ module AgentHookSpecHelper codex_events = AmberCLI::Agent::HookEvents.new codex_events.session_start = [legacy_hook_group("session", nil)] codex_events.pre_tool_use = [legacy_hook_group("pre", AmberCLI::Agent::MergeAgentHooksIntoSettings::CODEX_PRE_MATCHER)] - codex_events.post_tool_use = [legacy_hook_group("post", AmberCLI::Agent::MergeAgentHooksIntoSettings::POST_MATCHER)] + codex_events.post_tool_use = [legacy_hook_group("post", "Edit|Write|MultiEdit|NotebookEdit")] codex_events.stop = [legacy_hook_group("stop", nil)] codex = AmberCLI::Agent::HookSettings.new codex.hooks = codex_events diff --git a/spec/commands/doctor_spec.cr b/spec/commands/doctor_spec.cr index 7e48c4c..d1b1cbd 100644 --- a/spec/commands/doctor_spec.cr +++ b/spec/commands/doctor_spec.cr @@ -21,7 +21,7 @@ module DoctorCommandSpecHelper events = AmberCLI::Agent::HookEvents.new events.session_start = [legacy_hook_group("session", nil)] events.pre_tool_use = [legacy_hook_group("pre", pre_matcher)] - events.post_tool_use = [legacy_hook_group("post", AmberCLI::Agent::MergeAgentHooksIntoSettings::POST_MATCHER)] + events.post_tool_use = [legacy_hook_group("post", "Edit|Write|MultiEdit|NotebookEdit")] events.stop = [legacy_hook_group("stop", nil)] settings = AmberCLI::Agent::HookSettings.new settings.hooks = events diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr index 25aeb0b..ffc564b 100644 --- a/spec/commands/setup_agent_spec.cr +++ b/spec/commands/setup_agent_spec.cr @@ -77,6 +77,22 @@ describe "amber setup:agent" do settings.should contain(".amber/amber-agent-hook\\\" pre") end + it "matches the PostToolUse tools used by each agent" do + claude = AmberCLI::Agent::HookSettings.from_json(AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", true).perform) + codex = AmberCLI::Agent::HookSettings.from_json(AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", false).perform) + + if claude_events = claude.hooks + claude_events.post_tool_use.map(&.matcher).should eq(["Edit|Write"]) + else + fail "Claude hooks were not generated" + end + if codex_events = codex.hooks + codex_events.post_tool_use.map(&.matcher).should eq(["apply_patch|Bash"]) + else + fail "Codex hooks were not generated" + end + end + it "installs the loop into a project without changing existing instructions twice" do SpecHelper.within_temp_directory do |project| Dir.mkdir_p("src") @@ -185,7 +201,7 @@ describe "amber setup:agent" do run_hook.call("pre", "{}", "0", "0")[0].should eq(0) edited_file = File.join(project, "src/my app.cr") - run_hook.call("post", {"tool_input" => {"file_path" => edited_file}}.to_json, "0", "0")[0].should eq(0) + run_hook.call("post", {"tool_name" => "Edit", "tool_input" => {"file_path" => edited_file}}.to_json, "0", "0")[0].should eq(0) calls = File.read(log) calls.should contain("compiler watch hold") calls.should contain("compiler tool format #{edited_file}") @@ -194,11 +210,11 @@ describe "amber setup:agent" do quoted_file = File.join(project, %(src/odd"file.cr)) File.write(quoted_file, "puts :ok\n") - run_hook.call("post", {"tool_input" => {"file_path" => quoted_file}}.to_json, "0", "0")[0].should eq(0) + run_hook.call("post", {"tool_name" => "Edit", "tool_input" => {"file_path" => quoted_file}}.to_json, "0", "0")[0].should eq(0) File.read(log).should contain("compiler tool format #{quoted_file}") previous_calls = File.read(log) - run_hook.call("post", {"tool_input" => {"file_path" => "README.md"}}.to_json, "0", "0")[0].should eq(0) + run_hook.call("post", {"tool_name" => "Edit", "tool_input" => {"file_path" => "README.md"}}.to_json, "0", "0")[0].should eq(0) File.read(log).should eq(previous_calls) another_file = File.join(project, "src/another_file.cr") @@ -210,9 +226,34 @@ describe "amber setup:agent" do codex_calls.should match(/compiler tool format .*\/src\/another_file\.cr/) codex_calls.should match(/lsp --check .*\/src\/another_file\.cr/) + added_file = File.join(File.realpath(project), "src/added_by_patch.cr") + File.write(added_file, "puts :added_by_patch\n") + add_patch = "*** Begin Patch\n*** Add File: src/added_by_patch.cr\n+puts :added_by_patch\n*** End Patch" + add_payload = {"tool_name" => "apply_patch", "tool_input" => {"command" => add_patch}}.to_json + run_hook.call("post", add_payload, "0", "0")[0].should eq(0) + add_calls = File.read(log) + add_calls.should contain("compiler tool format #{added_file}") + add_calls.should contain("lsp --check #{added_file}") + + bash_file = File.join(File.realpath(project), "src/bash_write.cr") + File.write(bash_file, "puts :bash_write\n") + bash_payload = {"tool_name" => "Bash", "tool_input" => {"command" => "cat > src/bash_write.cr <<'EOF'\nputs :bash_write\nEOF"}}.to_json + run_hook.call("post", bash_payload, "0", "0")[0].should eq(0) + bash_calls = File.read(log) + bash_calls.should contain("compiler tool format #{bash_file}") + bash_calls.should contain("lsp --check #{bash_file}") + + tee_file = File.join(File.realpath(project), "src/tee_write.cr") + File.write(tee_file, "puts :tee_write\n") + tee_payload = {"tool_name" => "Bash", "tool_input" => {"command" => "printf 'puts :tee_write' | tee src/tee_write.cr"}}.to_json + run_hook.call("post", tee_payload, "0", "0")[0].should eq(0) + tee_calls = File.read(log) + tee_calls.should contain("compiler tool format #{tee_file}") + tee_calls.should contain("lsp --check #{tee_file}") + diagnostic_output = IO::Memory.new diagnostic_errors = IO::Memory.new - diagnostic_status = Process.run(hook, ["post"], input: IO::Memory.new({"tool_input" => {"file_path" => edited_file}}.to_json), + diagnostic_status = Process.run(hook, ["post"], input: IO::Memory.new({"tool_name" => "Edit", "tool_input" => {"file_path" => edited_file}}.to_json), output: diagnostic_output, error: diagnostic_errors, env: {"PATH" => path, "TEST_COMMAND_LOG" => log, "TEST_LSP_EXIT" => "1"}) diagnostic_status.exit_code.should eq(2) @@ -335,7 +376,7 @@ describe "amber setup:agent" do File.write(compiler, "#!/bin/sh\nprintf 'compiler %s\\n' \"$*\" >> \"$TEST_COMMAND_LOG\"\nif [ \"${TEST_FORMAT_EXIT:-0}\" -ne 0 ]; then\n echo 'format failed'\n exit \"$TEST_FORMAT_EXIT\"\nfi\n") File.chmod(compiler, 0o755) hook = File.join(project, ".amber/amber-agent-hook") - payload = {"tool_input" => {"file_path" => "src/my_app.cr"}}.to_json + payload = {"tool_name" => "Edit", "tool_input" => {"file_path" => "src/my_app.cr"}}.to_json File.write("src/other.cr", "puts :other\n") patch = "*** Begin Patch\n*** Update File: src/my_app.cr\n*** Update File: src/other.cr\n*** End Patch" multiple_files_payload = {"tool_name" => "apply_patch", "tool_input" => {"command" => patch}}.to_json diff --git a/src/amber_cli/agent/hook_settings.cr b/src/amber_cli/agent/hook_settings.cr index 09dc135..bab3c68 100644 --- a/src/amber_cli/agent/hook_settings.cr +++ b/src/amber_cli/agent/hook_settings.cr @@ -84,7 +84,8 @@ module AmberCLI::Agent class MergeAgentHooksIntoSettings CLAUDE_PRE_MATCHER = "Edit|Write|Bash" CODEX_PRE_MATCHER = "apply_patch|Bash" - POST_MATCHER = "Edit|Write|MultiEdit|NotebookEdit" + CLAUDE_POST_MATCHER = "Edit|Write" + CODEX_POST_MATCHER = "apply_patch|Bash" CLAUDE_COMMAND_PREFIX = %("$CLAUDE_PROJECT_DIR"/.amber/amber-agent-hook ) CODEX_COMMAND_PREFIX = %(sh -c 'root=$(git rev-parse --show-toplevel 2>/dev/null) && exec "$root/.amber/amber-agent-hook" ) @@ -96,6 +97,10 @@ module AmberCLI::Agent end end + def self.post_matcher_for(is_claude_settings : Bool) : String + is_claude_settings ? CLAUDE_POST_MATCHER : CODEX_POST_MATCHER + end + def initialize(@existing_json : String, @is_claude_settings : Bool = true) end @@ -106,7 +111,7 @@ module AmberCLI::Agent remove_stale_generated_hooks(hooks) add_hook(hooks.session_start, nil, self.class.generated_command_for("session", @is_claude_settings)) add_hook(hooks.pre_tool_use, pre_matcher, self.class.generated_command_for("pre", @is_claude_settings)) - add_hook(hooks.post_tool_use, POST_MATCHER, self.class.generated_command_for("post", @is_claude_settings)) + add_hook(hooks.post_tool_use, self.class.post_matcher_for(@is_claude_settings), self.class.generated_command_for("post", @is_claude_settings)) add_hook(hooks.stop, nil, self.class.generated_command_for("stop", @is_claude_settings)) settings.hooks = hooks settings.to_pretty_json + "\n" @@ -119,10 +124,14 @@ module AmberCLI::Agent {"post", events.post_tool_use}, {"stop", events.stop}, }.each do |mode, groups| - legacy_command = ".amber/amber-agent-hook #{mode}" + generated_commands = [ + ".amber/amber-agent-hook #{mode}", + self.class.generated_command_for(mode, true), + self.class.generated_command_for(mode, false), + ] groups.each do |group| group.hooks = group.hooks.reject do |handler| - handler.type == "command" && handler.command == legacy_command + handler.type == "command" && generated_commands.any? { |command| handler.command == command } end end end diff --git a/src/amber_cli/commands/doctor.cr b/src/amber_cli/commands/doctor.cr index b8cffcd..f7c9e37 100644 --- a/src/amber_cli/commands/doctor.cr +++ b/src/amber_cli/commands/doctor.cr @@ -164,7 +164,7 @@ module AmberCLI::Commands matcher = is_claude_settings ? AmberCLI::Agent::MergeAgentHooksIntoSettings::CLAUDE_PRE_MATCHER : AmberCLI::Agent::MergeAgentHooksIntoSettings::CODEX_PRE_MATCHER {events.pre_tool_use, matcher} when "post" - {events.post_tool_use, AmberCLI::Agent::MergeAgentHooksIntoSettings::POST_MATCHER} + {events.post_tool_use, AmberCLI::Agent::MergeAgentHooksIntoSettings.post_matcher_for(is_claude_settings)} else {events.stop, nil} end diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook index 913b16f..69d2c66 100644 --- a/src/amber_cli/templates/agent/amber-agent-hook +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -509,8 +509,62 @@ print_setup_refusal() { printf '%s\n' "Do not edit Crystal files yet. Tell the user to run \`amber setup:agent\` in this project, then continue." } -pre_write_contains_crystal() { +# Keep preflight and post-edit file detection on one payload parser. +changed_cr_files() { awk ' + function add_cr_path(path) { + sub(/\r$/, "", path) + if (path ~ /[.]cr$/ && !seen[path]++) print path + } + function flush_token() { + if (token == "") return + if (expect_redirect) { + add_cr_path(token) + expect_redirect = 0 + } else if (tee_targets && token !~ /^-/) { + add_cr_path(token) + } + if (is_command_position) { + if (token == "tee") tee_targets = 1 + is_command_position = 0 + } + token = "" + } + function scan_bash_command(command, command_index, character, quote, escaped, single_quote, double_quote) { + single_quote = sprintf("%c", 39) + double_quote = sprintf("%c", 34) + is_command_position = 1 + for (command_index = 1; command_index <= length(command); command_index++) { + character = substr(command, command_index, 1) + if (escaped) { + token = token character + escaped = 0 + } else if (quote != "") { + if (character == quote) quote = "" + else if (character == "\\" && quote == double_quote) escaped = 1 + else token = token character + } else if (character == single_quote || character == double_quote) { + quote = character + } else if (character == "\\") { + escaped = 1 + } else if (character ~ /[[:space:]]/) { + flush_token() + } else if (character == "|" || character == "&" || character == ";") { + flush_token() + tee_targets = 0 + is_command_position = 1 + } else if (character == ">") { + flush_token() + expect_redirect = 1 + } else if (character == "<") { + flush_token() + expect_redirect = 0 + } else { + token = token character + } + } + flush_token() + } function string_value( character, escaped, result) { result = ""; escaped = 0 while (position <= length(payload)) { @@ -545,25 +599,19 @@ pre_write_contains_crystal() { } if ((tool_name == "Edit" || tool_name == "Write") && direct_path ~ /[.]cr$/) { - print "crystal-write" - exit - } - if (tool_name == "apply_patch" && index(command_text, "*** Begin Patch") > 0) { + add_cr_path(direct_path) + } else if (tool_name == "apply_patch" && index(command_text, "*** Begin Patch") > 0) { count = split(command_text, patch_lines, "\n") for (line_number = 1; line_number <= count; line_number++) { line = patch_lines[line_number] if (line ~ /^\*\*\* (Update|Add) File: /) { sub(/^\*\*\* (Update|Add) File: /, "", line) sub(/\r$/, "", line) - if (line ~ /[.]cr$/) { print "crystal-write"; exit } + add_cr_path(line) } } - } - if (tool_name == "Bash") { - gsub(/\"/, "", command_text) - gsub(/\047/, "", command_text) - if (command_text ~ /(^|[[:space:];|&])tee[[:space:]][^;|&]*[.]cr([[:space:];|&]|$)/ || - command_text ~ />[^;|&]*[.]cr([[:space:];|&]|$)/) print "crystal-write" + } else if (tool_name == "Bash") { + scan_bash_command(command_text) } } ' @@ -589,7 +637,7 @@ case "${1:-}" in exit 0 ;; pre) - if [ -n "$(pre_write_contains_crystal)" ]; then + if [ -n "$(changed_cr_files)" ]; then if ! run_setup_check; then print_setup_refusal >&2 exit 2 @@ -603,58 +651,7 @@ case "${1:-}" in fi ;; post) - # Claude supplies file_path; Codex supplies an apply_patch command. - # Parse JSON strings and patch headers without another runtime. - edited_files=$(awk ' - function string_value( character, escaped, result) { - result = ""; escaped = 0 - while (position <= length(payload)) { - character = substr(payload, position++, 1) - if (escaped) { - if (character == "n") result = result "\n" - else if (character == "r") result = result "\r" - else if (character == "t") result = result "\t" - else if (character == "u") return "" - else result = result character - escaped = 0; continue - } - if (character == "\\") { escaped = 1; continue } - if (character == "\"") return result - result = result character - } - return "" - } - { payload = payload $0 "\n" } - END { - position = 1 - while (position <= length(payload)) { - if (substr(payload, position++, 1) != "\"") continue - key = string_value() - while (substr(payload, position, 1) ~ /[[:space:]]/) position++ - if (substr(payload, position++, 1) != ":") continue - while (substr(payload, position, 1) ~ /[[:space:]]/) position++ - if (substr(payload, position++, 1) != "\"") continue - value = string_value() - if (key == "file_path") direct_path = value - else if (key == "path" && direct_path == "") direct_path = value - else if (key == "tool_name") tool_name = value - else if (key == "command") patch = value - } - if (tool_name == "apply_patch" && index(patch, "*** Begin Patch") > 0) { - count = split(patch, lines, "\n") - for (line_number = 1; line_number <= count; line_number++) { - line = lines[line_number] - if (line ~ /^\*\*\* (Update|Add) File: /) { - sub(/^\*\*\* (Update|Add) File: /, "", line) - sub(/\r$/, "", line) - if (!seen[line]++) print line - } - } - } else if (direct_path != "") { - print direct_path - } - } - ') + edited_files=$(changed_cr_files) [ -n "$edited_files" ] || exit 0 lsp=$(find_lsp) || lsp='' printf '%s\n' "$edited_files" | while IFS= read -r edited_file; do From 277c6ba84048323bc3840b412e57bf210e54b53a Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 22:41:19 -0400 Subject: [PATCH 38/69] fix(amber-cli): block post hook without amber-lsp --- spec/commands/setup_agent_spec.cr | 10 ++++--- .../templates/agent/amber-agent-hook | 29 ++++++++++--------- 2 files changed, 22 insertions(+), 17 deletions(-) diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr index ffc564b..ca62405 100644 --- a/spec/commands/setup_agent_spec.cr +++ b/spec/commands/setup_agent_spec.cr @@ -361,7 +361,7 @@ describe "amber setup:agent" do end end - it "formats without an LSP and finds a project LSP when configured one is absent" do + it "blocks post without an LSP and finds a configured LSP when one is available" do SpecHelper.within_temp_directory do |project| Dir.mkdir_p("src") File.write("shard.yml", "name: my_app\ntargets:\n my_app:\n main: src/my_app.cr\n") @@ -384,9 +384,11 @@ describe "amber setup:agent" do errors = IO::Memory.new missing_status = Process.run(hook, ["post"], input: IO::Memory.new(multiple_files_payload), output: IO::Memory.new, error: errors, env: environment) - missing_status.exit_code.should eq(0) - errors.to_s.scan(/amber-lsp unavailable/).size.should eq(1) - File.read(log).scan(/compiler tool format /).size.should eq(2) + missing_status.exit_code.should eq(2) + errors.to_s.should contain("The amber-lsp binary was not found at amber-lsp or on PATH.") + errors.to_s.should contain("Tell the user to run `amber setup:agent` in this project, then continue.") + errors.to_s.should_not contain("amber-lsp unavailable") + File.file?(log).should be_false project_lsp = File.join(project, "bin/amber-lsp") Dir.mkdir_p("bin") diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook index 69d2c66..f26ce35 100644 --- a/src/amber_cli/templates/agent/amber-agent-hook +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -509,6 +509,12 @@ print_setup_refusal() { printf '%s\n' "Do not edit Crystal files yet. Tell the user to run \`amber setup:agent\` in this project, then continue." } +print_post_refusal() { + requested_binary=${AMBER_LSP_BIN:-amber-lsp} + printf 'The amber-lsp binary was not found at %s or on PATH.\n' "$requested_binary" >&2 + printf '%s\n' "Tell the user to run \`amber setup:agent\` in this project, then continue." >&2 +} + # Keep preflight and post-edit file detection on one payload parser. changed_cr_files() { awk ' @@ -654,6 +660,10 @@ case "${1:-}" in edited_files=$(changed_cr_files) [ -n "$edited_files" ] || exit 0 lsp=$(find_lsp) || lsp='' + if [ -z "$lsp" ]; then + print_post_refusal + exit 2 + fi printf '%s\n' "$edited_files" | while IFS= read -r edited_file; do case "$edited_file" in *.cr) ;; @@ -677,20 +687,13 @@ case "${1:-}" in exit 2 fi [ -z "$format_output" ] || printf '%s\n' "$format_output" - if [ -n "$lsp" ]; then - diagnostics=$("$lsp" --check "$edited_file" 2>&1) - diagnostic_status=$? - if [ "$diagnostic_status" -ne 0 ]; then - [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" >&2 - exit 2 - fi - [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" - else - if [ "${warned_missing_lsp:-0}" -eq 0 ]; then - echo 'amber-agent-hook: amber-lsp unavailable; skipping file diagnostics' >&2 - warned_missing_lsp=1 - fi + diagnostics=$("$lsp" --check "$edited_file" 2>&1) + diagnostic_status=$? + if [ "$diagnostic_status" -ne 0 ]; then + [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" >&2 + exit 2 fi + [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" done exit $? ;; From 504a7150352dddd670902d1e1013595085285920 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 23:00:22 -0400 Subject: [PATCH 39/69] fix(lsp): index all dependency docs entries --- spec/amber_lsp/lookup/api_cards_spec.cr | 4 + .../lookup/build_layered_api_index_spec.cr | 37 ++++ spec/amber_lsp/lookup/index_cache_spec.cr | 8 + .../cards/library/.amber-lsp/api/example.yml | 3 + .../api_lookup/cards/library/src/example.cr | 4 + .../api_lookup/cards/library/src/ui.cr | 6 + src/amber_lsp/lookup/api_cards.cr | 17 ++ .../lookup/build_crystal_docs_json.cr | 4 +- .../lookup/build_layered_api_index.cr | 187 ++++++++++++++---- src/amber_lsp/lookup/index_cache.cr | 14 +- src/amber_lsp/lookup/index_models.cr | 13 ++ .../lookup/merge_api_index_layers.cr | 64 ++++++ .../lookup/resolve_api_index_docs_entries.cr | 53 +++++ 13 files changed, 370 insertions(+), 44 deletions(-) create mode 100644 spec/fixtures/api_lookup/cards/library/src/example.cr create mode 100644 spec/fixtures/api_lookup/cards/library/src/ui.cr create mode 100644 src/amber_lsp/lookup/merge_api_index_layers.cr create mode 100644 src/amber_lsp/lookup/resolve_api_index_docs_entries.cr diff --git a/spec/amber_lsp/lookup/api_cards_spec.cr b/spec/amber_lsp/lookup/api_cards_spec.cr index 950dc23..13eb2b4 100644 --- a/spec/amber_lsp/lookup/api_cards_spec.cr +++ b/spec/amber_lsp/lookup/api_cards_spec.cr @@ -21,6 +21,9 @@ describe AmberLSP::Lookup::LoadAPICards do "fixture_project" => ["fixture_project_docs"], "example" => ["example_docs"], }) + cards.docs_entries_by_library.should eq({ + "example" => ["src/example.cr", "src/ui.cr"], + }) cards.matching_notes("FixtureAPI::User#id").map(&.text).should contain( "The ID getter is generated from the persisted table column." ) @@ -79,6 +82,7 @@ private def with_api_card_project(&) FileUtils.cp_r(File.join(fixture_root, "project", ".amber-lsp"), project) Dir.mkdir_p(File.join(project, "lib", "example")) FileUtils.cp_r(File.join(fixture_root, "library", ".amber-lsp"), File.join(project, "lib", "example")) + FileUtils.cp_r(File.join(fixture_root, "library", "src"), File.join(project, "lib", "example")) File.write(File.join(project, "shard.lock"), "shards:\n example:\n version: 0.3.5\n") yield project end diff --git a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr index f28a1d9..69be2c8 100644 --- a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr +++ b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr @@ -27,6 +27,15 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do File.join(project_root, "lib", "tiny", "src", "tiny.cr"), "require \"sibling\"\nmodule Tiny\n class Client\n def ping : Bool\n true\n end\n\n def sibling_token : Sibling::Token\n Sibling::Token.new\n end\n end\nend\n", ) + File.write( + File.join(project_root, "lib", "tiny", "src", "ui.cr"), + "module Tiny\n module UI\n class Label\n def text=(new_text : String) : String\n new_text\n end\n end\n end\nend\n", + ) + File.write( + File.join(project_root, "lib", "tiny", "src", "components.cr"), + "module Tiny\n class FrameworkRegistry\n end\n\n class Client\n def ping : Bool\n true\n end\n end\nend\n", + ) + File.write(File.join(project_root, "lib", "tiny", "src", "broken.cr"), "module Tiny\n class Broken\n") File.write( File.join(project_root, "lib", "sibling", "src", "sibling.cr"), "module Sibling\n class Token\n end\nend\n", @@ -71,8 +80,16 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do tiny_client.not_nil!.list_of_instance_methods.first.name.should eq("ping") if client = tiny_client client.list_of_instance_methods.find { |method| method.name == "sibling_token" }.should_not be_nil + client.list_of_instance_methods.count { |method| method.name == "ping" }.should eq(1) client.location_path.should eq(File.join(project_root, "lib", "tiny", "src", "tiny.cr")) end + tiny_types = library_layer.not_nil!.layer.not_nil!.list_of_types + tiny_types.any? { |type| type.name == "Tiny::UI::Label" }.should be_true + tiny_types.any? { |type| type.name == "Tiny::FrameworkRegistry" }.should be_true + failed_entries = library_layer.not_nil!.layer.not_nil!.to_json + failed_entries.should contain("src/broken.cr") + failed_entries.should contain("entry_failures") + failed_entries.should contain("crystal-alpha docs failed") standard_types = standard_library_layer.not_nil!.layer.not_nil!.list_of_types dir_type = standard_types.find { |type| type.name == "Dir" } @@ -87,4 +104,24 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do Dir.children(File.join(cache_root, "docs-workspaces")).should be_empty end end + + it "uses API-card docs entries instead of automatic library entries" do + with_tempdir do |root| + source_root = File.join(root, "src") + Dir.mkdir_p(File.join(source_root, "nested")) + ["tiny.cr", "ui.cr", "components.cr"].each do |filename| + File.write(File.join(source_root, filename), "module Tiny\nend\n") + end + + automatic_entries = AmberLSP::Lookup::ResolveAPIIndexDocsEntries.new(root, "tiny").perform + explicit_entries = AmberLSP::Lookup::ResolveAPIIndexDocsEntries.new( + root, + "tiny", + ["src/ui.cr", "src/tiny.cr"], + ).perform + + automatic_entries.should eq(["src/tiny.cr", "src/components.cr", "src/ui.cr"]) + explicit_entries.should eq(["src/tiny.cr", "src/ui.cr"]) + end + end end diff --git a/spec/amber_lsp/lookup/index_cache_spec.cr b/spec/amber_lsp/lookup/index_cache_spec.cr index b23f2be..a7d3781 100644 --- a/spec/amber_lsp/lookup/index_cache_spec.cr +++ b/spec/amber_lsp/lookup/index_cache_spec.cr @@ -51,11 +51,19 @@ describe "AmberLSP::Lookup::CalculateLibraryLayerKey#perform" do changed_version = AmberLSP::Lookup::CalculateLibraryLayerKey.new("grant", "0.23.5", ["grant_docs"], "Crystal 1.21.0").perform changed_flags = AmberLSP::Lookup::CalculateLibraryLayerKey.new("grant", "0.23.4+git.commit.abc123", [] of String, "Crystal 1.21.0").perform changed_compiler = AmberLSP::Lookup::CalculateLibraryLayerKey.new("grant", "0.23.4+git.commit.abc123", ["grant_docs"], "Crystal 1.21.1").perform + changed_entries = AmberLSP::Lookup::CalculateLibraryLayerKey.new( + "grant", + "0.23.4+git.commit.abc123", + ["grant_docs"], + "Crystal 1.21.0", + ["src/grant.cr", "src/ui.cr"], + ).perform same_key.should eq(first_key) changed_version.should_not eq(first_key) changed_flags.should_not eq(first_key) changed_compiler.should_not eq(first_key) + changed_entries.should_not eq(first_key) end end diff --git a/spec/fixtures/api_lookup/cards/library/.amber-lsp/api/example.yml b/spec/fixtures/api_lookup/cards/library/.amber-lsp/api/example.yml index 5530e42..90989ea 100644 --- a/spec/fixtures/api_lookup/cards/library/.amber-lsp/api/example.yml +++ b/spec/fixtures/api_lookup/cards/library/.amber-lsp/api/example.yml @@ -3,6 +3,9 @@ library: example applies_to: "~> 0.3.0" docs_flags: - example_docs +docs_entries: + - src/example.cr + - src/ui.cr notes: - symbol: Example::Widget#id text: This getter follows the library's primary key. diff --git a/spec/fixtures/api_lookup/cards/library/src/example.cr b/spec/fixtures/api_lookup/cards/library/src/example.cr new file mode 100644 index 0000000..50c2823 --- /dev/null +++ b/spec/fixtures/api_lookup/cards/library/src/example.cr @@ -0,0 +1,4 @@ +module Example + class Widget + end +end diff --git a/spec/fixtures/api_lookup/cards/library/src/ui.cr b/spec/fixtures/api_lookup/cards/library/src/ui.cr new file mode 100644 index 0000000..2efe310 --- /dev/null +++ b/spec/fixtures/api_lookup/cards/library/src/ui.cr @@ -0,0 +1,6 @@ +module Example + module UI + class View + end + end +end diff --git a/src/amber_lsp/lookup/api_cards.cr b/src/amber_lsp/lookup/api_cards.cr index 4bf6d37..b689dea 100644 --- a/src/amber_lsp/lookup/api_cards.cr +++ b/src/amber_lsp/lookup/api_cards.cr @@ -29,6 +29,7 @@ module AmberLSP::Lookup getter library : String getter applies_to : String getter docs_flags : Array(String) = [] of String + getter docs_entries : Array(String) = [] of String getter notes : Array(APICardNote) = [] of APICardNote getter error_hints : Array(APICardErrorHint) = [] of APICardErrorHint end @@ -67,6 +68,16 @@ module AmberLSP::Lookup list_of_cards end + def docs_entries_by_library : Hash(String, Array(String)) + list_of_entries = {} of String => Array(String) + @list_of_cards.each do |loaded_card| + next if loaded_card.card.docs_entries.empty? + + list_of_entries[loaded_card.card.library] = loaded_card.card.docs_entries + end + list_of_entries + end + def matching_notes(query : String) : Array(APICardNote) query_method_name = query.split(/[.#]/).last? @list_of_cards.flat_map do |loaded_card| @@ -238,6 +249,12 @@ module AmberLSP::Lookup raise ArgumentError.new("invalid docs flag #{flag.inspect}") unless flag.matches?(/\A[A-Za-z_][A-Za-z0-9_]*\z/) end + card.docs_entries.each do |entry| + path = Path[entry] + raise ArgumentError.new("docs_entries paths must be relative to the shard root") if entry.empty? || path.absolute? + raise ArgumentError.new("docs_entries paths cannot traverse outside the shard root") if entry.split(/[\\\/]/).includes?("..") + end + card.notes.each do |note| raise ArgumentError.new("each note needs a symbol and text") if note.symbol.empty? || note.text.empty? end diff --git a/src/amber_lsp/lookup/build_crystal_docs_json.cr b/src/amber_lsp/lookup/build_crystal_docs_json.cr index 7dc6279..1bfb122 100644 --- a/src/amber_lsp/lookup/build_crystal_docs_json.cr +++ b/src/amber_lsp/lookup/build_crystal_docs_json.cr @@ -7,7 +7,7 @@ module AmberLSP::Lookup def initialize( @compiler_path : String, @working_directory : String, - @entrypoint : String, + @list_of_entrypoints : Array(String), @project_name : String, @project_version : String, @docs_flags : Array(String), @@ -28,7 +28,7 @@ module AmberLSP::Lookup arguments << "-D" arguments << flag end - arguments << @entrypoint + arguments.concat(@list_of_entrypoints) begin status = Process.run( diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr index 4cdff76..6124ba4 100644 --- a/src/amber_lsp/lookup/build_layered_api_index.cr +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -7,7 +7,9 @@ require "yaml" require "./build_crystal_docs_json" require "./api_cards" require "./index_cache" +require "./merge_api_index_layers" require "./normalize_crystal_docs" +require "./resolve_api_index_docs_entries" require "./source_models" module AmberLSP::Lookup @@ -52,13 +54,13 @@ module AmberLSP::Lookup # :nodoc: struct CrystalDocsWorkspace getter root_path : String - getter entrypoint : String + getter list_of_entrypoints : Array(String) getter crystal_cache_path : String getter source_path_mappings : Hash(String, String) def initialize( @root_path : String, - @entrypoint : String, + @list_of_entrypoints : Array(String), @crystal_cache_path : String, @source_path_mappings : Hash(String, String), ) @@ -77,7 +79,9 @@ module AmberLSP::Lookup def perform : Array(CachedAPIIndexLayer) manifest = read_project_manifest - card_flags = LoadAPICards.new(@root_path).perform.docs_flags_by_library + cards = LoadAPICards.new(@root_path).perform + card_flags = cards.docs_flags_by_library + card_entries = cards.docs_entries_by_library list_of_docs_flags = card_flags.merge(@list_of_docs_flags_by_library) identity = DetectCrystalAlpha.new(@compiler_command).perform cache = APIIndexCache.new(@cache_root, @root_path) @@ -95,7 +99,7 @@ module AmberLSP::Lookup project_key, @root_path, project_flags, - project_entrypoint, + [project_entrypoint], @root_path, {} of String => String, ) @@ -128,21 +132,34 @@ module AmberLSP::Lookup next end - entrypoint = find_library_entrypoint(library_root, library_name) - unless entrypoint + docs_entries : Array(String)? = nil + begin + docs_entries = ResolveAPIIndexDocsEntries.new( + library_root, + library_name, + card_entries[library_name]?, + ).perform + rescue ex : APIIndexBuildError list_of_layers << CachedAPIIndexLayer.new( nil, "unavailable", - "Locked shard #{library_name} has no Crystal entrypoint under lib/#{library_name}/src", + "Locked shard #{library_name} has no usable Crystal docs entries: #{ex.message}", "library", library_name, locked_shard.version, ) next end + resolved_docs_entries = docs_entries || raise APIIndexBuildError.new("Could not resolve docs entries for #{library_name}") docs_flags = docs_flags_for(library_name, list_of_docs_flags) - layer_key = CalculateLibraryLayerKey.new(library_name, locked_shard.version, docs_flags, identity.version).perform + layer_key = CalculateLibraryLayerKey.new( + library_name, + locked_shard.version, + docs_flags, + identity.version, + resolved_docs_entries, + ).perform list_of_layers << build_or_load_layer( cache, identity, @@ -151,7 +168,7 @@ module AmberLSP::Lookup layer_key, library_root, docs_flags, - entrypoint, + resolved_docs_entries, library_root, {} of String => String, ) @@ -203,17 +220,6 @@ module AmberLSP::Lookup first_source.sub(@root_path + "/", "") end - private def find_library_entrypoint(library_root : String, library_name : String) : String? - source_root = File.join(library_root, "src") - return nil unless File.directory?(source_root) - - conventional_name = library_name.gsub('-', '_') - conventional_entrypoint = File.join(source_root, "#{conventional_name}.cr") - return conventional_entrypoint if File.file?(conventional_entrypoint) - - Dir.glob(File.join(source_root, "*.cr")).sort.first? || Dir.glob(File.join(source_root, "**", "*.cr")).sort.first? - end - private def docs_flags_for(library_name : String, list_of_docs_flags : Hash(String, Array(String))) : Array(String) list_of_docs_flags[library_name]?.try(&.sort) || [] of String end @@ -226,7 +232,7 @@ module AmberLSP::Lookup layer_key : String, source_root : String, docs_flags : Array(String), - entrypoint : String, + list_of_entrypoints : Array(String), working_directory : String, environment_overrides : Hash(String, String), ) : CachedAPIIndexLayer @@ -236,38 +242,33 @@ module AmberLSP::Lookup workspace_root : String? = nil begin docs_working_directory = working_directory - docs_entrypoint = relative_entrypoint(entrypoint, working_directory) + list_of_docs_entrypoints = list_of_entrypoints.map { |entrypoint| relative_entrypoint(entrypoint, working_directory) } docs_environment_overrides = environment_overrides source_path_mappings = {} of String => String if layer_kind == "library" - workspace = create_library_docs_workspace(source_root, entrypoint) + workspace = create_library_docs_workspace(File.join(source_root, "src"), list_of_entrypoints) workspace_root = workspace.root_path docs_working_directory = workspace.root_path - docs_entrypoint = workspace.entrypoint + list_of_docs_entrypoints = workspace.list_of_entrypoints docs_environment_overrides = environment_overrides.merge({"CRYSTAL_CACHE_DIR" => workspace.crystal_cache_path}) source_path_mappings = workspace.source_path_mappings end - docs_json = BuildCrystalDocsJSON.new( + layer = build_layer_from_docs_entries( identity.executable_path, docs_working_directory, - docs_entrypoint, + list_of_docs_entrypoints, layer_name, - "0", - docs_flags, - docs_environment_overrides, - ).perform - layer = NormalizeCrystalDocs.new( - docs_json, source_root, layer_kind, - layer_name, layer_key, docs_flags, + docs_environment_overrides, docs_working_directory, source_path_mappings, - ).perform + list_of_entrypoints, + ) cache.write_layer(layer) CachedAPIIndexLayer.new(layer, "fresh") rescue ex : APIIndexBuildError | JSON::ParseException @@ -283,7 +284,113 @@ module AmberLSP::Lookup end end - private def create_library_docs_workspace(source_root : String, entrypoint : String) : CrystalDocsWorkspace + private def build_layer_from_docs_entries( + compiler_path : String, + working_directory : String, + list_of_docs_entrypoints : Array(String), + layer_name : String, + source_root : String, + layer_kind : String, + layer_key : String, + docs_flags : Array(String), + environment_overrides : Hash(String, String), + docs_working_directory : String, + source_path_mappings : Hash(String, String), + list_of_source_entrypoints : Array(String), + ) : APIIndexLayer + combined_failure_reason : String? = nil + begin + docs_json = BuildCrystalDocsJSON.new( + compiler_path, + working_directory, + list_of_docs_entrypoints, + layer_name, + "0", + docs_flags, + environment_overrides, + ).perform + return normalize_docs_layer( + docs_json, + source_root, + layer_kind, + layer_name, + layer_key, + docs_flags, + docs_working_directory, + source_path_mappings, + ) + rescue ex : APIIndexBuildError | JSON::ParseException + # A combined docs invocation can fail because one entry is unavailable on its own. + combined_failure_reason = ex.message + end + + list_of_normalized_layers = [] of APIIndexLayer + list_of_entry_failures = [] of APIIndexEntryFailure + list_of_docs_entrypoints.each_with_index do |entrypoint, index| + source_entrypoint = list_of_source_entrypoints[index] + begin + docs_json = BuildCrystalDocsJSON.new( + compiler_path, + working_directory, + [entrypoint], + layer_name, + "0", + docs_flags, + environment_overrides, + ).perform + list_of_normalized_layers << normalize_docs_layer( + docs_json, + source_root, + layer_kind, + layer_name, + layer_key, + docs_flags, + docs_working_directory, + source_path_mappings, + ) + rescue ex : APIIndexBuildError | JSON::ParseException + list_of_entry_failures << APIIndexEntryFailure.new( + source_entrypoint, + ex.message || "crystal-alpha docs failed for #{source_entrypoint}", + ) + end + end + + main_entrypoint = list_of_source_entrypoints.first? || "" + if main_failure = list_of_entry_failures.find { |failure| failure.entry_path == main_entrypoint } + raise APIIndexBuildError.new("Main docs entry #{main_failure.entry_path} failed: #{main_failure.error}") + end + if list_of_normalized_layers.empty? + reason = combined_failure_reason || "Combined and individual Crystal docs entries failed" + raise APIIndexBuildError.new(reason) + end + + MergeAPIIndexLayers.new(list_of_normalized_layers, list_of_entry_failures).perform + end + + private def normalize_docs_layer( + docs_json : String, + source_root : String, + layer_kind : String, + layer_name : String, + layer_key : String, + docs_flags : Array(String), + docs_working_directory : String, + source_path_mappings : Hash(String, String), + ) : APIIndexLayer + NormalizeCrystalDocs.new( + docs_json, + source_root, + layer_kind, + layer_name, + layer_key, + docs_flags, + docs_working_directory, + source_path_mappings, + ).perform + end + + private def create_library_docs_workspace(source_root : String, list_of_entrypoints : Array(String)) : CrystalDocsWorkspace source_path = File.expand_path(source_root) library_root = File.dirname(source_path) workspace_root = "" @@ -299,8 +406,10 @@ module AmberLSP::Lookup crystal_cache_path = File.join(workspace_root, "crystal-cache") Dir.mkdir_p(crystal_cache_path) - source_relative_entrypoint = entrypoint.sub(source_path + "/", "") - workspace_entrypoint = File.join("src", source_relative_entrypoint) + list_of_workspace_entrypoints = list_of_entrypoints.map do |entrypoint| + relative_entrypoint = entrypoint.sub(library_root + "/", "") + File.join("src", relative_entrypoint.sub("src/", "")) + end source_path_mappings = { File.join(workspace_root, "src") => source_path, File.join(workspace_root, "lib") => project_lib_path, @@ -308,7 +417,7 @@ module AmberLSP::Lookup CrystalDocsWorkspace.new( workspace_root, - workspace_entrypoint, + list_of_workspace_entrypoints, crystal_cache_path, source_path_mappings, ) @@ -334,7 +443,7 @@ module AmberLSP::Lookup docs_json = BuildCrystalDocsJSON.new( identity.executable_path, File.dirname(source_root), - File.join("src", "docs_main.cr"), + [File.join("src", "docs_main.cr")], "crystal", "0", docs_flags, diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr index b25b311..3a36c9e 100644 --- a/src/amber_lsp/lookup/index_cache.cr +++ b/src/amber_lsp/lookup/index_cache.cr @@ -9,13 +9,19 @@ module AmberLSP::Lookup end class CalculateProjectLayerKey - def initialize(@root_path : String, @docs_flags : Array(String), @compiler_version : String) + def initialize( + @root_path : String, + @docs_flags : Array(String), + @compiler_version : String, + @docs_entries : Array(String) = [] of String, + ) end def perform : String root_path = File.expand_path(@root_path) list_of_index_paths = source_paths(root_path) - key_material = ["project", @compiler_version, @docs_flags.sort.join("\0")] + key_material = ["project", "docs-entries-v1", @compiler_version, @docs_flags.sort.join("\0")] + key_material.concat(@docs_entries) list_of_index_paths.each do |path| relative_path = path.sub(root_path + "/", "") @@ -47,18 +53,20 @@ module AmberLSP::Lookup @locked_version_or_commit : String, @docs_flags : Array(String), @compiler_version : String, + @docs_entries : Array(String) = [] of String, ) end def perform : String key_material = [ "library", - "docs-workspace-v1", + "docs-workspace-multi-entry-v2", @library_name, @locked_version_or_commit, @docs_flags.sort.join("\0"), @compiler_version, ] + key_material.concat(@docs_entries) Digest::SHA256.hexdigest(key_material.join("\0")) end end diff --git a/src/amber_lsp/lookup/index_models.cr b/src/amber_lsp/lookup/index_models.cr index bf5aa29..53302f8 100644 --- a/src/amber_lsp/lookup/index_models.cr +++ b/src/amber_lsp/lookup/index_models.cr @@ -204,6 +204,17 @@ module AmberLSP::Lookup end end + # :nodoc: + struct APIIndexEntryFailure + include JSON::Serializable + + getter entry_path : String + getter error : String + + def initialize(@entry_path : String, @error : String) + end + end + # :nodoc: struct APIIndexLayer include JSON::Serializable @@ -214,6 +225,7 @@ module AmberLSP::Lookup getter root_path : String getter docs_flags : Array(String) getter list_of_types : Array(APIIndexType) + getter list_of_entry_failures : Array(APIIndexEntryFailure) = [] of APIIndexEntryFailure def initialize( @layer_kind : String, @@ -222,6 +234,7 @@ module AmberLSP::Lookup @root_path : String, @docs_flags : Array(String), @list_of_types : Array(APIIndexType), + @list_of_entry_failures : Array(APIIndexEntryFailure) = [] of APIIndexEntryFailure, ) end end diff --git a/src/amber_lsp/lookup/merge_api_index_layers.cr b/src/amber_lsp/lookup/merge_api_index_layers.cr new file mode 100644 index 0000000..765770e --- /dev/null +++ b/src/amber_lsp/lookup/merge_api_index_layers.cr @@ -0,0 +1,64 @@ +require "set" + +require "./index_models" + +module AmberLSP::Lookup + class MergeAPIIndexLayers + def initialize( + @list_of_layers : Array(APIIndexLayer), + @list_of_entry_failures : Array(APIIndexEntryFailure) = [] of APIIndexEntryFailure, + ) + end + + def perform : APIIndexLayer + first_layer = @list_of_layers.first? || raise APIIndexBuildError.new("No Crystal docs entries succeeded") + list_of_types_by_name = {} of String => APIIndexType + list_of_type_names = [] of String + + @list_of_layers.each do |layer| + layer.list_of_types.each do |type| + if existing_type = list_of_types_by_name[type.name]? + list_of_types_by_name[type.name] = merge_types(existing_type, type) + else + list_of_type_names << type.name + list_of_types_by_name[type.name] = type + end + end + end + + APIIndexLayer.new( + first_layer.layer_kind, + first_layer.layer_name, + first_layer.layer_key, + first_layer.root_path, + first_layer.docs_flags, + list_of_type_names.map { |name| list_of_types_by_name[name] }, + @list_of_entry_failures, + ) + end + + private def merge_types(first_type : APIIndexType, next_type : APIIndexType) : APIIndexType + APIIndexType.new( + first_type.name, + first_type.kind, + first_type.abstract? || next_type.abstract?, + (first_type.list_of_ancestor_names + next_type.list_of_ancestor_names).uniq, + (first_type.list_of_included_module_names + next_type.list_of_included_module_names).uniq, + (first_type.list_of_extended_module_names + next_type.list_of_extended_module_names).uniq, + first_type.location_path.empty? ? next_type.location_path : first_type.location_path, + first_type.location_line == 0 ? next_type.location_line : first_type.location_line, + first_type.source_layer, + deduplicate_methods(first_type.list_of_instance_methods + next_type.list_of_instance_methods), + deduplicate_methods(first_type.list_of_class_methods + next_type.list_of_class_methods), + deduplicate_methods(first_type.list_of_macros + next_type.list_of_macros), + ) + end + + private def deduplicate_methods(list_of_methods : Array(APIIndexMethod)) : Array(APIIndexMethod) + seen = Set(Tuple(String, String, String)).new + list_of_methods.select do |method| + seen.add?({method.owner, method.name, method.args_string}) + end + end + end +end diff --git a/src/amber_lsp/lookup/resolve_api_index_docs_entries.cr b/src/amber_lsp/lookup/resolve_api_index_docs_entries.cr new file mode 100644 index 0000000..272d305 --- /dev/null +++ b/src/amber_lsp/lookup/resolve_api_index_docs_entries.cr @@ -0,0 +1,53 @@ +module AmberLSP::Lookup + class ResolveAPIIndexDocsEntries + def initialize( + @root_path : String, + @library_name : String, + @list_of_overrides : Array(String)? = nil, + ) + end + + def perform : Array(String) + root_path = File.expand_path(@root_path) + if list_of_overrides = @list_of_overrides + return normalize_overrides(root_path, list_of_overrides) + end + + source_root = File.join(root_path, "src") + list_of_entries = Dir.glob(File.join(source_root, "*.cr")).sort + if list_of_entries.empty? + list_of_entries = Dir.glob(File.join(source_root, "**", "*.cr")).sort + end + raise APIIndexBuildError.new("No Crystal entrypoint found under #{source_root}") if list_of_entries.empty? + + conventional_entry = File.join(source_root, "#{@library_name.gsub('-', '_')}.cr") + main_entry = list_of_entries.includes?(conventional_entry) ? conventional_entry : list_of_entries.first + ([main_entry] + list_of_entries.reject { |entry| entry == main_entry }) + .map { |entry| entry.sub(root_path + "/", "") } + end + + private def normalize_overrides(root_path : String, list_of_overrides : Array(String)) : Array(String) + raise APIIndexBuildError.new("docs_entries must contain at least one entry") if list_of_overrides.empty? + + list_of_entries = list_of_overrides.map do |entry| + path = Path[entry] + raise APIIndexBuildError.new("docs_entries paths must be relative to the shard root") if entry.empty? || path.absolute? + raise APIIndexBuildError.new("docs_entries paths cannot traverse outside the shard root") if entry.split(/[\\\/]/).includes?("..") + + expanded_path = File.expand_path(entry, root_path) + unless expanded_path.starts_with?(root_path + "/") && File.file?(expanded_path) + raise APIIndexBuildError.new("docs entry #{entry.inspect} does not exist under #{root_path}") + end + + entry + end + + conventional_entry = File.join("src", "#{@library_name.gsub('-', '_')}.cr") + if list_of_entries.includes?(conventional_entry) + return [conventional_entry] + list_of_entries.reject { |entry| entry == conventional_entry } + end + + list_of_entries + end + end +end From 99e8e245fd65a8976443c33658fe12575481acb1 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Mon, 5 Oct 2026 23:04:14 -0400 Subject: [PATCH 40/69] fix(lsp): index source entries in library projects --- .../lookup/build_layered_api_index_spec.cr | 65 +++++++++++++++++++ .../lookup/build_layered_api_index.cr | 27 +++++++- 2 files changed, 89 insertions(+), 3 deletions(-) diff --git a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr index 69be2c8..45487b8 100644 --- a/spec/amber_lsp/lookup/build_layered_api_index_spec.cr +++ b/spec/amber_lsp/lookup/build_layered_api_index_spec.cr @@ -124,4 +124,69 @@ describe "AmberLSP::Lookup::BuildLayeredAPIIndex#perform" do explicit_entries.should eq(["src/tiny.cr", "src/ui.cr"]) end end + + it "indexes all top-level source entries in a project library without a targets main" do + with_tempdir do |project_root| + cache_root = File.join(project_root, "cache") + source_root = File.join(project_root, "src") + Dir.mkdir_p(source_root) + File.write(File.join(project_root, "shard.yml"), "name: fixture_library\nversion: 1.0.0\n") + File.write( + File.join(source_root, "fixture_library.cr"), + "module FixtureLibrary\n class Client\n def ping : Bool\n true\n end\n end\nend\n", + ) + File.write( + File.join(source_root, "ui.cr"), + "module FixtureLibrary\n module UI\n class Label\n def text=(new_text : String) : String\n new_text\n end\n end\n end\nend\n", + ) + File.write(File.join(source_root, "components.cr"), "module FixtureLibrary\n class FrameworkRegistry\n end\nend\n") + + list_of_layers = AmberLSP::Lookup::BuildLayeredAPIIndex.new(project_root, cache_root).perform + project_layer = list_of_layers.find { |cached| cached.layer_kind == "project" } || raise "Expected a project API layer" + project_layer.freshness.should eq("fresh") + project_api_layer = project_layer.layer || raise "Expected project docs to produce an API layer" + project_types = project_api_layer.list_of_types + project_types.any? { |type| type.name == "FixtureLibrary::UI::Label" }.should be_true + project_types.any? { |type| type.name == "FixtureLibrary::FrameworkRegistry" }.should be_true + end + end + + it "uses the project API card docs entries for a project library and keys that selection" do + with_tempdir do |project_root| + cache_root = File.join(project_root, "cache") + source_root = File.join(project_root, "src") + card_root = File.join(project_root, ".amber-lsp", "api") + Dir.mkdir_p(source_root) + Dir.mkdir_p(card_root) + File.write(File.join(project_root, "shard.yml"), "name: fixture_library\nversion: 1.0.0\n") + File.write(File.join(source_root, "fixture_library.cr"), "module FixtureLibrary\n class Client\n end\nend\n") + File.write(File.join(source_root, "ui.cr"), "module FixtureLibrary\n class UIComponent\n end\nend\n") + File.write(File.join(source_root, "components.cr"), "module FixtureLibrary\n class FrameworkRegistry\n end\nend\n") + card_path = File.join(card_root, "fixture_library.yml") + File.write( + card_path, + "card_version: 1\nlibrary: fixture_library\napplies_to: 1.0.0\ndocs_entries:\n - src/ui.cr\n - src/fixture_library.cr\n", + ) + + first_layers = AmberLSP::Lookup::BuildLayeredAPIIndex.new(project_root, cache_root).perform + first_project_layer = first_layers.find { |cached| cached.layer_kind == "project" } || raise "Expected a project API layer" + first_project_layer.freshness.should eq("fresh") + first_api_layer = first_project_layer.layer || raise "Expected project docs to produce an API layer" + first_types = first_api_layer.list_of_types + first_types.any? { |type| type.name == "FixtureLibrary::UIComponent" }.should be_true + first_types.any? { |type| type.name == "FixtureLibrary::FrameworkRegistry" }.should be_false + + File.write( + card_path, + "card_version: 1\nlibrary: fixture_library\napplies_to: 1.0.0\ndocs_entries:\n - src/fixture_library.cr\n", + ) + second_layers = AmberLSP::Lookup::BuildLayeredAPIIndex.new(project_root, cache_root).perform + second_project_layer = second_layers.find { |cached| cached.layer_kind == "project" } || raise "Expected a project API layer" + second_project_layer.freshness.should eq("fresh") + second_project_layer.layer_key.should_not eq(first_project_layer.layer_key) + second_api_layer = second_project_layer.layer || raise "Expected updated project docs to produce an API layer" + second_types = second_api_layer.list_of_types + second_types.any? { |type| type.name == "FixtureLibrary::UIComponent" }.should be_false + end + end end diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr index 6124ba4..c4ee3cc 100644 --- a/src/amber_lsp/lookup/build_layered_api_index.cr +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -88,8 +88,13 @@ module AmberLSP::Lookup list_of_layers = [] of CachedAPIIndexLayer project_name = manifest.name.empty? ? File.basename(@root_path) : manifest.name project_flags = docs_flags_for(project_name, list_of_docs_flags) - project_key = CalculateProjectLayerKey.new(@root_path, project_flags, identity.version).perform - project_entrypoint = find_project_entrypoint(manifest) + project_entrypoints = project_docs_entrypoints(manifest, project_name, card_entries) + project_key = CalculateProjectLayerKey.new( + @root_path, + project_flags, + identity.version, + project_entrypoints, + ).perform list_of_layers << build_or_load_layer( cache, @@ -99,7 +104,7 @@ module AmberLSP::Lookup project_key, @root_path, project_flags, - [project_entrypoint], + project_entrypoints, @root_path, {} of String => String, ) @@ -220,6 +225,22 @@ module AmberLSP::Lookup first_source.sub(@root_path + "/", "") end + private def project_docs_entrypoints( + manifest : ProjectShardManifest, + project_name : String, + docs_entries_by_library : Hash(String, Array(String)), + ) : Array(String) + has_target_main = manifest.targets.values.any? { |target| !target.main.empty? } + has_source_files = !Dir.glob(File.join(@root_path, "src", "**", "*.cr")).empty? + return [find_project_entrypoint(manifest)] if has_target_main || !has_source_files + + ResolveAPIIndexDocsEntries.new( + @root_path, + project_name, + docs_entries_by_library[project_name]?, + ).perform + end + private def docs_flags_for(library_name : String, list_of_docs_flags : Hash(String, Array(String))) : Array(String) list_of_docs_flags[library_name]?.try(&.sort) || [] of String end From 48f2a0c9da9babd59026e24eecc0187a9069d01b Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 01:07:25 -0400 Subject: [PATCH 41/69] fix(lsp): resolve receiver-specific return types --- spec/amber_lsp/lookup/lookup_command_spec.cr | 2 + .../lookup/lsp_lookup_controller_spec.cr | 35 +++++++++++ .../lookup/resolve_api_query_spec.cr | 24 +++++++ .../api_lookup/crystal_docs_return_types.json | 45 ++++++++++++++ src/amber_lsp/lookup/index_cache.cr | 5 +- src/amber_lsp/lookup/index_models.cr | 35 +++++++++-- src/amber_lsp/lookup/resolve_api_query.cr | 33 +++++++--- .../lookup/resolve_api_return_type.cr | 62 +++++++++++++++++++ 8 files changed, 227 insertions(+), 14 deletions(-) create mode 100644 spec/fixtures/api_lookup/crystal_docs_return_types.json create mode 100644 src/amber_lsp/lookup/resolve_api_return_type.cr diff --git a/spec/amber_lsp/lookup/lookup_command_spec.cr b/spec/amber_lsp/lookup/lookup_command_spec.cr index 0795a2c..df00343 100644 --- a/spec/amber_lsp/lookup/lookup_command_spec.cr +++ b/spec/amber_lsp/lookup/lookup_command_spec.cr @@ -118,6 +118,8 @@ describe AmberLSP::Lookup::RunLookupCommand do json["status"].as_s.should eq("found") json["freshness"].as_s.should eq("fresh") json["entries"].as_a.first["name"].as_s.should eq("id") + json["entries"].as_a.first["declared_return_type"].as_s.should eq("Int64 | ::Nil") + json["entries"].as_a.first["resolved_return_type"].as_s.should eq("Int64 | Nil") json["verification_status"].as_s.should eq("present") json["verification_elapsed_milliseconds"].as_i.should be >= 0 json["card_notes"].as_a.size.should eq(1) diff --git a/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr b/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr index b528e79..14e6e30 100644 --- a/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr +++ b/spec/amber_lsp/lookup/lsp_lookup_controller_spec.cr @@ -2,6 +2,41 @@ require "../spec_helper" require "../../../src/amber_lsp/lookup/normalize_crystal_docs" describe AmberLSP::Controller do + it "shows a receiver-resolved return type in hover markdown" do + with_lsp_lookup_project do |project, service| + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_return_types.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + project, + "project", + "fixture_return_types", + "fixture-key", + [] of String, + ).perform + service.seed(project, [AmberLSP::Lookup::CachedAPIIndexLayer.new(layer, "fresh")]) + + source_path = File.join(project, "src", "models", "user.cr") + File.write(source_path, "FixtureTypes::Post.find\n") + uri = "file://#{source_path}" + controller = AmberLSP::Controller.new(service) + server = AmberLSP::Server.new(IO::Memory.new, IO::Memory.new) + initialize_lookup_controller(controller, server, project) + response = controller.handle( + lsp_lookup_request(7, "textDocument/hover", { + "textDocument" => {"uri" => uri}, + "position" => {"line" => 0, "character" => 21}, + }), + server, + ) || raise "Expected receiver-resolved hover response" + hover_response = AmberLSP::Lookup::LSPHoverResponse.from_json(response) + hover = hover_response.result || raise "Expected hover contents" + + hover.contents.markdown_value.should contain( + "FixtureTypes::ClassMethods.find(id : Int64) : FixtureTypes::Post | Nil (declared: self?) (class method via extend)", + ) + end + end + it "serves workspace symbols, hover, and definitions through the controller" do with_lsp_lookup_project do |project, service| controller = AmberLSP::Controller.new(service) diff --git a/spec/amber_lsp/lookup/resolve_api_query_spec.cr b/spec/amber_lsp/lookup/resolve_api_query_spec.cr index 7ae6df9..01e222b 100644 --- a/spec/amber_lsp/lookup/resolve_api_query_spec.cr +++ b/spec/amber_lsp/lookup/resolve_api_query_spec.cr @@ -42,6 +42,17 @@ describe "AmberLSP::Lookup::ResolveAPIQuery#perform" do result.list_of_methods.first.owner.should eq("Grant::Query::Builder(Model)") end + it "resolves self and nilable return types against the queried receiver" do + result = resolve_return_type_fixture_query("FixtureTypes::Post.find") + + entry = result.list_of_methods.first + entry.declared_return_type.should eq("self?") + entry.resolved_return_type.should eq("FixtureTypes::Post | Nil") + entry.lookup_signature.should eq( + "FixtureTypes::ClassMethods.find(id : Int64) : FixtureTypes::Post | Nil (declared: self?) (class method via extend)", + ) + end + it "returns a type summary for a type-only query, including a generic base-name match" do result = resolve_fixture_query("FixtureAPI::Team") generic_result = resolve_fixture_query("Grant::Query::Builder") @@ -82,3 +93,16 @@ private def resolve_fixture_query(query : String) : AmberLSP::Lookup::APIResolut ).perform AmberLSP::Lookup::ResolveAPIQuery.new(query, [layer]).perform end + +private def resolve_return_type_fixture_query(query : String) : AmberLSP::Lookup::APIResolution + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_return_types.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + "/project", + "project", + "fixture_return_types", + "fixture-key", + [] of String, + ).perform + AmberLSP::Lookup::ResolveAPIQuery.new(query, [layer]).perform +end diff --git a/spec/fixtures/api_lookup/crystal_docs_return_types.json b/spec/fixtures/api_lookup/crystal_docs_return_types.json new file mode 100644 index 0000000..83b1fce --- /dev/null +++ b/spec/fixtures/api_lookup/crystal_docs_return_types.json @@ -0,0 +1,45 @@ +{ + "repository_name": "fixture_return_types", + "program": { + "kind": "module", + "name": "FixtureTypes", + "full_name": "FixtureTypes", + "types": [ + { + "kind": "class", + "name": "Post", + "full_name": "FixtureTypes::Post", + "locations": [{"filename": "src/post.cr", "line_number": 1}], + "extended_modules": [{"kind": "module", "name": "ClassMethods", "full_name": "FixtureTypes::ClassMethods"}], + "instance_methods": [ + { + "name": "reload", + "args_string": "() : self", + "location": {"filename": "src/post.cr", "line_number": 5}, + "def": {"name": "reload", "return_type": "self"} + }, + { + "name": "id", + "args_string": "() : Int64 | ::Nil", + "location": {"filename": "src/post.cr", "line_number": 9}, + "def": {"name": "id", "return_type": "Int64 | ::Nil"} + } + ] + }, + { + "kind": "module", + "name": "ClassMethods", + "full_name": "FixtureTypes::ClassMethods", + "locations": [{"filename": "src/class_methods.cr", "line_number": 1}], + "instance_methods": [ + { + "name": "find", + "args_string": "(id : Int64) : self?", + "location": {"filename": "src/class_methods.cr", "line_number": 4}, + "def": {"name": "find", "return_type": "self?"} + } + ] + } + ] + } +} diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr index 3a36c9e..2dec431 100644 --- a/src/amber_lsp/lookup/index_cache.cr +++ b/src/amber_lsp/lookup/index_cache.cr @@ -20,7 +20,7 @@ module AmberLSP::Lookup def perform : String root_path = File.expand_path(@root_path) list_of_index_paths = source_paths(root_path) - key_material = ["project", "docs-entries-v1", @compiler_version, @docs_flags.sort.join("\0")] + key_material = ["project", "docs-entries-v1", "resolved-return-types-v1", @compiler_version, @docs_flags.sort.join("\0")] key_material.concat(@docs_entries) list_of_index_paths.each do |path| @@ -61,6 +61,7 @@ module AmberLSP::Lookup key_material = [ "library", "docs-workspace-multi-entry-v2", + "resolved-return-types-v1", @library_name, @locked_version_or_commit, @docs_flags.sort.join("\0"), @@ -76,7 +77,7 @@ module AmberLSP::Lookup end def perform : String - key_material = ["stdlib", @compiler_version, @docs_flags.sort.join("\0")] + key_material = ["stdlib", "resolved-return-types-v1", @compiler_version, @docs_flags.sort.join("\0")] Digest::SHA256.hexdigest(key_material.join("\0")) end end diff --git a/src/amber_lsp/lookup/index_models.cr b/src/amber_lsp/lookup/index_models.cr index 53302f8..cf3a121 100644 --- a/src/amber_lsp/lookup/index_models.cr +++ b/src/amber_lsp/lookup/index_models.cr @@ -105,7 +105,8 @@ module AmberLSP::Lookup getter name : String getter method_kind : String getter args_string : String - getter return_type : String + getter declared_return_type : String + getter resolved_return_type : String getter doc_line : String? = nil getter source_path : String getter source_line : Int32 @@ -118,14 +119,20 @@ module AmberLSP::Lookup @name : String, @method_kind : String, @args_string : String, - @return_type : String, + @declared_return_type : String, @doc_line : String?, @source_path : String, @source_line : Int32, @source_layer : String, @abstract : Bool, @macro : Bool, + resolved_return_type : String? = nil, ) + @resolved_return_type = resolved_return_type || @declared_return_type + end + + def return_type : String + declared_return_type end def as_extended_class_method : APIIndexMethod @@ -134,13 +141,31 @@ module AmberLSP::Lookup name, "extended_class", args_string, - return_type, + declared_return_type, + doc_line, + source_path, + source_line, + source_layer, + abstract?, + macro?, + resolved_return_type, + ) + end + + def with_resolved_return_type(resolved_type : String) : APIIndexMethod + APIIndexMethod.new( + owner, + name, + method_kind, + args_string, + declared_return_type, doc_line, source_path, source_line, source_layer, abstract?, macro?, + resolved_type, ) end @@ -158,7 +183,9 @@ module AmberLSP::Lookup end def lookup_signature : String - "#{lookup_label} : #{return_type.gsub("::Nil", "Nil")}#{extension_annotation}" + signature = "#{lookup_label} : #{resolved_return_type}" + signature += " (declared: #{declared_return_type})" if declared_return_type != resolved_return_type + "#{signature}#{extension_annotation}" end private def lookup_argument_list : String diff --git a/src/amber_lsp/lookup/resolve_api_query.cr b/src/amber_lsp/lookup/resolve_api_query.cr index 2c24ce0..d901926 100644 --- a/src/amber_lsp/lookup/resolve_api_query.cr +++ b/src/amber_lsp/lookup/resolve_api_query.cr @@ -1,6 +1,7 @@ require "set" require "./index_models" +require "./resolve_api_return_type" module AmberLSP::Lookup struct APIResolution @@ -54,11 +55,11 @@ module AmberLSP::Lookup return unknown_result(query) if matching_types.empty? methods = methods_named(matching_types, method_name, "instance") - return found_result(query, "instance_method", methods) unless methods.empty? + return found_result(query, "instance_method", methods, receiver_type(type_name, matching_types)) unless methods.empty? ancestor_types_for(matching_types).each do |ancestor_type| methods = methods_named([ancestor_type], method_name, "instance") - return found_result(query, "instance_method", methods) unless methods.empty? + return found_result(query, "instance_method", methods, receiver_type(type_name, matching_types)) unless methods.empty? end unknown_result(query) @@ -69,19 +70,19 @@ module AmberLSP::Lookup return unknown_result(query) if matching_types.empty? methods = methods_named(matching_types, method_name, "class") - return found_result(query, "class_method", methods) unless methods.empty? + return found_result(query, "class_method", methods, receiver_type(type_name, matching_types)) unless methods.empty? ancestor_types = ancestor_types_for(matching_types) ancestor_types.each do |ancestor_type| methods = methods_named([ancestor_type], method_name, "class") - return found_result(query, "class_method", methods) unless methods.empty? + return found_result(query, "class_method", methods, receiver_type(type_name, matching_types)) unless methods.empty? end methods = extended_module_methods(matching_types + ancestor_types, method_name) - return found_result(query, "class_method", methods) unless methods.empty? + return found_result(query, "class_method", methods, receiver_type(type_name, matching_types)) unless methods.empty? macro_methods = methods_named(matching_types, method_name, "macro") - return found_result(query, "class_method", macro_methods) unless macro_methods.empty? + return found_result(query, "class_method", macro_methods, receiver_type(type_name, matching_types)) unless macro_methods.empty? unknown_result(query) end @@ -206,8 +207,24 @@ module AmberLSP::Lookup output.to_s end - private def found_result(query : String, resolution_kind : String, methods : Array(APIIndexMethod)) : APIResolution - APIResolution.new(query, resolution_kind, nil, methods) + private def receiver_type(type_name : String, matching_types : Array(APIIndexType)) : String + normalized_name = normalize_type_name(type_name) + return normalized_name if normalized_name.includes?('(') + + matching_types.first?.try(&.name) || normalized_name + end + + private def found_result( + query : String, + resolution_kind : String, + methods : Array(APIIndexMethod), + receiver_type : String? = nil, + ) : APIResolution + resolved_methods = methods.map do |method| + resolved_return_type = ResolveAPIIndexReturnType.new(method.declared_return_type, receiver_type).perform + method.with_resolved_return_type(resolved_return_type) + end + APIResolution.new(query, resolution_kind, nil, resolved_methods) end private def unknown_result(query : String) : APIResolution diff --git a/src/amber_lsp/lookup/resolve_api_return_type.cr b/src/amber_lsp/lookup/resolve_api_return_type.cr new file mode 100644 index 0000000..3f0ae3d --- /dev/null +++ b/src/amber_lsp/lookup/resolve_api_return_type.cr @@ -0,0 +1,62 @@ +module AmberLSP::Lookup + class ResolveAPIIndexReturnType + def initialize(@declared_return_type : String, @receiver_type : String?) + end + + def perform : String + return "unknown" if @declared_return_type == "unknown" + + resolved_type = @declared_return_type + if receiver_type = @receiver_type + resolved_type = resolved_type.gsub(/\bself\b/) { receiver_type } + end + + resolved_type = expand_nilable_types(resolved_type) + resolved_type = resolved_type.gsub("::Nil", "Nil") + resolved_type = resolved_type.gsub(/\s+or\s+/, " | ") + normalize_union_order(resolved_type) + end + + private def expand_nilable_types(return_type : String) : String + return_type.gsub(/([A-Za-z_][A-Za-z0-9_:]*(?:\([^?]*\))?)\?/) do |match| + "#{match.byte_slice(0, match.bytesize - 1)} | Nil" + end + end + + private def normalize_union_order(return_type : String) : String + list_of_types = [] of String + current_type = String::Builder.new + nesting_depth = 0 + + return_type.each_char do |character| + case character + when '(' then nesting_depth += 1 + when ')' then nesting_depth -= 1 if nesting_depth > 0 + when '|' + if nesting_depth == 0 + append_type(list_of_types, current_type) + current_type = String::Builder.new + next + end + end + current_type << character + end + append_type(list_of_types, current_type) + + nil_type_present = list_of_types.any? { |type| type == "Nil" } + return list_of_types.join(" | ") unless nil_type_present + + list_of_types.reject! { |type| type == "Nil" } + list_of_types << "Nil" + list_of_types.join(" | ") + end + + private def append_type(list_of_types : Array(String), current_type : String::Builder) : Nil + type = current_type.to_s.strip + return if type.empty? + return if list_of_types.includes?(type) + + list_of_types << type + end + end +end From 755f4e413eb22430e5b9673946d523fc51227d79 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 01:14:44 -0400 Subject: [PATCH 42/69] fix(lsp): bind generic lookup receivers --- .../lookup/resolve_api_query_spec.cr | 36 +++++++++ .../api_lookup/crystal_docs_return_types.json | 80 +++++++++++++++++++ src/amber_lsp/lookup/parse_api_type_name.cr | 52 ++++++++++++ src/amber_lsp/lookup/resolve_api_query.cr | 29 +++---- .../lookup/resolve_api_return_type.cr | 64 ++++++++++++++- 5 files changed, 243 insertions(+), 18 deletions(-) create mode 100644 src/amber_lsp/lookup/parse_api_type_name.cr diff --git a/spec/amber_lsp/lookup/resolve_api_query_spec.cr b/spec/amber_lsp/lookup/resolve_api_query_spec.cr index 01e222b..395ef3a 100644 --- a/spec/amber_lsp/lookup/resolve_api_query_spec.cr +++ b/spec/amber_lsp/lookup/resolve_api_query_spec.cr @@ -53,6 +53,42 @@ describe "AmberLSP::Lookup::ResolveAPIQuery#perform" do ) end + it "parses generic receivers and binds each owner type parameter" do + builder_result = resolve_return_type_fixture_query("Grant::Query::Builder(Post)#first") + builder_first = builder_result.list_of_methods.find { |method| method.args_string == "()" } + builder_first_entry = builder_first || raise "Expected the no-argument first overload" + builder_first_entry.resolved_return_type.should eq("Post | Nil") + + bare_builder_result = resolve_return_type_fixture_query("Grant::Query::Builder#first") + bare_first = bare_builder_result.list_of_methods.find { |method| method.args_string == "()" } + bare_first_entry = bare_first || raise "Expected the bare generic first overload" + bare_first_entry.resolved_return_type.should eq("Model | Nil") + + lock_result = resolve_return_type_fixture_query("Grant::Query::Builder(Post)#lock") + lock_result.list_of_methods.first.resolved_return_type.should eq("Grant::Query::Builder(Post)") + + collection_result = resolve_return_type_fixture_query("Grant::AssociationCollection(User, Post)#target") + collection_result.list_of_methods.first.resolved_return_type.should eq("Post | Nil") + + owner_result = resolve_return_type_fixture_query("Grant::AssociationCollection(User, Post)#owner") + owner_result.list_of_methods.first.resolved_return_type.should eq("User") + + generic_collection_result = resolve_return_type_fixture_query("Grant::Collection(Post)#first") + generic_collection_result.list_of_methods.first.resolved_return_type.should eq("Post | Nil") + + loaded_result = resolve_return_type_fixture_query("Grant::LoadedAssociationCollection(User, Post)#clear") + loaded_result.list_of_methods.first.resolved_return_type.should eq( + "Grant::LoadedAssociationCollection(User, Post)", + ) + + nested_loaded_result = resolve_return_type_fixture_query( + "Grant::LoadedAssociationCollection(User, Grant::Query::Builder(Post))#target", + ) + nested_loaded_result.list_of_methods.first.resolved_return_type.should eq( + "Grant::Query::Builder(Post) | Nil", + ) + end + it "returns a type summary for a type-only query, including a generic base-name match" do result = resolve_fixture_query("FixtureAPI::Team") generic_result = resolve_fixture_query("Grant::Query::Builder") diff --git a/spec/fixtures/api_lookup/crystal_docs_return_types.json b/spec/fixtures/api_lookup/crystal_docs_return_types.json index 83b1fce..58bfb31 100644 --- a/spec/fixtures/api_lookup/crystal_docs_return_types.json +++ b/spec/fixtures/api_lookup/crystal_docs_return_types.json @@ -39,6 +39,86 @@ "def": {"name": "find", "return_type": "self?"} } ] + }, + { + "kind": "class", + "name": "Builder", + "full_name": "Grant::Query::Builder(Model)", + "locations": [{"filename": "lib/grant/src/grant/query/builder.cr", "line_number": 1}], + "instance_methods": [ + { + "name": "first", + "args_string": "() : Model | ::Nil", + "location": {"filename": "lib/grant/src/grant/query/builder.cr", "line_number": 20}, + "def": {"name": "first", "return_type": "Model | ::Nil"} + }, + { + "name": "first", + "args_string": "(n : Int32) : Array(Model)", + "location": {"filename": "lib/grant/src/grant/query/builder.cr", "line_number": 30}, + "def": {"name": "first", "return_type": "Array(Model)"} + }, + { + "name": "lock", + "args_string": "(enabled : Bool) : self", + "location": {"filename": "lib/grant/src/grant/query/locking.cr", "line_number": 40}, + "def": {"name": "lock", "return_type": "self"} + } + ] + }, + { + "kind": "class", + "name": "Collection", + "full_name": "Grant::Collection(T)", + "locations": [{"filename": "lib/grant/src/grant/collection.cr", "line_number": 1}], + "instance_methods": [ + { + "name": "first", + "args_string": "() : T?", + "location": {"filename": "lib/grant/src/grant/collection.cr", "line_number": 10}, + "def": {"name": "first", "return_type": "T?"} + } + ] + }, + { + "kind": "class", + "name": "AssociationCollection", + "full_name": "Grant::AssociationCollection(Owner, Target)", + "locations": [{"filename": "lib/grant/src/grant/association_collection.cr", "line_number": 1}], + "instance_methods": [ + { + "name": "owner", + "args_string": "() : Owner", + "location": {"filename": "lib/grant/src/grant/association_collection.cr", "line_number": 10}, + "def": {"name": "owner", "return_type": "Owner"} + }, + { + "name": "target", + "args_string": "() : Target?", + "location": {"filename": "lib/grant/src/grant/association_collection.cr", "line_number": 15}, + "def": {"name": "target", "return_type": "Target?"} + } + ] + }, + { + "kind": "class", + "name": "LoadedAssociationCollection", + "full_name": "Grant::LoadedAssociationCollection(Owner, Target)", + "locations": [{"filename": "lib/grant/src/grant/loaded_association_collection.cr", "line_number": 1}], + "instance_methods": [ + { + "name": "clear", + "args_string": "() : self", + "location": {"filename": "lib/grant/src/grant/loaded_association_collection.cr", "line_number": 10}, + "def": {"name": "clear", "return_type": "self"} + }, + { + "name": "target", + "args_string": "() : Target?", + "location": {"filename": "lib/grant/src/grant/loaded_association_collection.cr", "line_number": 15}, + "def": {"name": "target", "return_type": "Target?"} + } + ] } ] } diff --git a/src/amber_lsp/lookup/parse_api_type_name.cr b/src/amber_lsp/lookup/parse_api_type_name.cr new file mode 100644 index 0000000..b82a22e --- /dev/null +++ b/src/amber_lsp/lookup/parse_api_type_name.cr @@ -0,0 +1,52 @@ +module AmberLSP::Lookup + struct ParsedAPITypeName + getter base_name : String + getter list_of_type_arguments : Array(String) + + def initialize(@base_name : String, @list_of_type_arguments : Array(String)) + end + end + + class ParseAPITypeName + def initialize(@name : String) + end + + def perform : ParsedAPITypeName + opening_index = @name.index('(') + return ParsedAPITypeName.new(@name.strip, [] of String) unless opening_index && @name.ends_with?(')') + + generic_arguments = @name.byte_slice(opening_index + 1, @name.bytesize - opening_index - 2) + ParsedAPITypeName.new( + @name.byte_slice(0, opening_index).strip, + split_type_arguments(generic_arguments), + ) + end + + private def split_type_arguments(generic_arguments : String) : Array(String) + list_of_arguments = [] of String + current_argument = String::Builder.new + nesting_depth = 0 + + generic_arguments.each_char do |character| + case character + when '(' then nesting_depth += 1 + when ')' then nesting_depth -= 1 if nesting_depth > 0 + when ',' + if nesting_depth == 0 + append_argument(list_of_arguments, current_argument) + current_argument = String::Builder.new + next + end + end + current_argument << character + end + append_argument(list_of_arguments, current_argument) + list_of_arguments + end + + private def append_argument(list_of_arguments : Array(String), current_argument : String::Builder) : Nil + argument = current_argument.to_s.strip + list_of_arguments << argument unless argument.empty? + end + end +end diff --git a/src/amber_lsp/lookup/resolve_api_query.cr b/src/amber_lsp/lookup/resolve_api_query.cr index d901926..27a0ac1 100644 --- a/src/amber_lsp/lookup/resolve_api_query.cr +++ b/src/amber_lsp/lookup/resolve_api_query.cr @@ -1,6 +1,7 @@ require "set" require "./index_models" +require "./parse_api_type_name" require "./resolve_api_return_type" module AmberLSP::Lookup @@ -190,28 +191,18 @@ module AmberLSP::Lookup end private def base_type_name(name : String) : String - output = String::Builder.new - generic_depth = 0 - - name.each_char do |character| - case character - when '(' - generic_depth += 1 - when ')' - generic_depth -= 1 if generic_depth > 0 - else - output << character unless generic_depth > 0 - end - end - - output.to_s + ParseAPITypeName.new(name).perform.base_name end private def receiver_type(type_name : String, matching_types : Array(APIIndexType)) : String normalized_name = normalize_type_name(type_name) return normalized_name if normalized_name.includes?('(') - matching_types.first?.try(&.name) || normalized_name + if matching_type = matching_types.first? + return matching_type.name + end + + normalized_name end private def found_result( @@ -221,7 +212,11 @@ module AmberLSP::Lookup receiver_type : String? = nil, ) : APIResolution resolved_methods = methods.map do |method| - resolved_return_type = ResolveAPIIndexReturnType.new(method.declared_return_type, receiver_type).perform + resolved_return_type = ResolveAPIIndexReturnType.new( + method.declared_return_type, + receiver_type, + method.owner, + ).perform method.with_resolved_return_type(resolved_return_type) end APIResolution.new(query, resolution_kind, nil, resolved_methods) diff --git a/src/amber_lsp/lookup/resolve_api_return_type.cr b/src/amber_lsp/lookup/resolve_api_return_type.cr index 3f0ae3d..d6b5d68 100644 --- a/src/amber_lsp/lookup/resolve_api_return_type.cr +++ b/src/amber_lsp/lookup/resolve_api_return_type.cr @@ -1,6 +1,12 @@ +require "./parse_api_type_name" + module AmberLSP::Lookup class ResolveAPIIndexReturnType - def initialize(@declared_return_type : String, @receiver_type : String?) + def initialize( + @declared_return_type : String, + @receiver_type : String?, + @owner_type_name : String? = nil, + ) end def perform : String @@ -11,12 +17,68 @@ module AmberLSP::Lookup resolved_type = resolved_type.gsub(/\bself\b/) { receiver_type } end + resolved_type = substitute_owner_type_parameters(resolved_type) resolved_type = expand_nilable_types(resolved_type) resolved_type = resolved_type.gsub("::Nil", "Nil") resolved_type = resolved_type.gsub(/\s+or\s+/, " | ") normalize_union_order(resolved_type) end + private def substitute_owner_type_parameters(return_type : String) : String + owner_type_name = @owner_type_name + receiver_type = @receiver_type + return return_type unless owner_type_name && receiver_type + + owner_type = ParseAPITypeName.new(owner_type_name).perform + receiver = ParseAPITypeName.new(receiver_type).perform + return return_type unless normalize_type_base(owner_type.base_name) == normalize_type_base(receiver.base_name) + return return_type unless owner_type.list_of_type_arguments.size == receiver.list_of_type_arguments.size + + bindings = {} of String => String + owner_type.list_of_type_arguments.each_with_index do |parameter_name, index| + argument = receiver.list_of_type_arguments[index]? + bindings[parameter_name] = argument if argument + end + substitute_type_tokens(return_type, bindings) + end + + private def normalize_type_base(name : String) : String + name.sub(/\A::/, "") + end + + private def substitute_type_tokens(return_type : String, bindings : Hash(String, String)) : String + output = String::Builder.new + current_token = String::Builder.new + + return_type.each_char do |character| + if type_token_character?(character) + current_token << character + else + append_substituted_token(output, current_token, bindings) + current_token = String::Builder.new + output << character + end + end + append_substituted_token(output, current_token, bindings) + output.to_s + end + + private def type_token_character?(character : Char) : Bool + (character >= 'A' && character <= 'Z') || + (character >= 'a' && character <= 'z') || + (character >= '0' && character <= '9') || + character == '_' || character == ':' + end + + private def append_substituted_token( + output : String::Builder, + current_token : String::Builder, + bindings : Hash(String, String), + ) : Nil + token = current_token.to_s + output << (bindings[token]? || token) + end + private def expand_nilable_types(return_type : String) : String return_type.gsub(/([A-Za-z_][A-Za-z0-9_:]*(?:\([^?]*\))?)\?/) do |match| "#{match.byte_slice(0, match.bytesize - 1)} | Nil" From e84e2ddd31bba018dad141f8931c48afe9dfd313 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 01:16:24 -0400 Subject: [PATCH 43/69] fix(lsp): resolve overload return types independently --- .../lookup/resolve_api_query_spec.cr | 13 ++++++++++ src/amber_lsp/lookup/resolve_api_query.cr | 11 ++------- ...olve_return_types_for_api_index_methods.cr | 24 +++++++++++++++++++ 3 files changed, 39 insertions(+), 9 deletions(-) create mode 100644 src/amber_lsp/lookup/resolve_return_types_for_api_index_methods.cr diff --git a/spec/amber_lsp/lookup/resolve_api_query_spec.cr b/spec/amber_lsp/lookup/resolve_api_query_spec.cr index 395ef3a..99a62fd 100644 --- a/spec/amber_lsp/lookup/resolve_api_query_spec.cr +++ b/spec/amber_lsp/lookup/resolve_api_query_spec.cr @@ -89,6 +89,19 @@ describe "AmberLSP::Lookup::ResolveAPIQuery#perform" do ) end + it "resolves each overload from that row's declared return type" do + resolution = resolve_return_type_fixture_query("Grant::Query::Builder(Post)#first") + entries = AmberLSP::Lookup::ResolveReturnTypesForAPIIndexMethods.new( + resolution.list_of_methods, + "Grant::Query::Builder(Post)", + ).perform + + entries.map(&.lookup_signature).should eq([ + "Grant::Query::Builder(Model)#first() : Post | Nil (declared: Model | ::Nil)", + "Grant::Query::Builder(Model)#first(n : Int32) : Array(Post) (declared: Array(Model))", + ]) + end + it "returns a type summary for a type-only query, including a generic base-name match" do result = resolve_fixture_query("FixtureAPI::Team") generic_result = resolve_fixture_query("Grant::Query::Builder") diff --git a/src/amber_lsp/lookup/resolve_api_query.cr b/src/amber_lsp/lookup/resolve_api_query.cr index 27a0ac1..4191097 100644 --- a/src/amber_lsp/lookup/resolve_api_query.cr +++ b/src/amber_lsp/lookup/resolve_api_query.cr @@ -2,7 +2,7 @@ require "set" require "./index_models" require "./parse_api_type_name" -require "./resolve_api_return_type" +require "./resolve_return_types_for_api_index_methods" module AmberLSP::Lookup struct APIResolution @@ -211,14 +211,7 @@ module AmberLSP::Lookup methods : Array(APIIndexMethod), receiver_type : String? = nil, ) : APIResolution - resolved_methods = methods.map do |method| - resolved_return_type = ResolveAPIIndexReturnType.new( - method.declared_return_type, - receiver_type, - method.owner, - ).perform - method.with_resolved_return_type(resolved_return_type) - end + resolved_methods = ResolveReturnTypesForAPIIndexMethods.new(methods, receiver_type).perform APIResolution.new(query, resolution_kind, nil, resolved_methods) end diff --git a/src/amber_lsp/lookup/resolve_return_types_for_api_index_methods.cr b/src/amber_lsp/lookup/resolve_return_types_for_api_index_methods.cr new file mode 100644 index 0000000..53cae8e --- /dev/null +++ b/src/amber_lsp/lookup/resolve_return_types_for_api_index_methods.cr @@ -0,0 +1,24 @@ +require "./index_models" +require "./resolve_api_return_type" + +module AmberLSP::Lookup + # :nodoc: + class ResolveReturnTypesForAPIIndexMethods + def initialize( + @list_of_methods : Array(APIIndexMethod), + @receiver_type : String?, + ) + end + + def perform : Array(APIIndexMethod) + @list_of_methods.map do |method| + resolved_return_type = ResolveAPIIndexReturnType.new( + method.declared_return_type, + @receiver_type, + method.owner, + ).perform + method.with_resolved_return_type(resolved_return_type) + end + end + end +end From a59644a7eb7c82b482036bef460db9c6944c6317 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 01:18:27 -0400 Subject: [PATCH 44/69] fix(lsp): preserve unknown return types --- spec/amber_lsp/lookup/resolve_api_query_spec.cr | 9 +++++++++ spec/fixtures/api_lookup/crystal_docs_return_types.json | 6 ++++++ src/amber_lsp/lookup/index_cache.cr | 6 +++--- src/amber_lsp/lookup/index_models.cr | 3 +++ src/amber_lsp/lookup/normalize_crystal_docs.cr | 7 ++++++- src/amber_lsp/lookup/resolve_api_return_type.cr | 2 +- 6 files changed, 28 insertions(+), 5 deletions(-) diff --git a/spec/amber_lsp/lookup/resolve_api_query_spec.cr b/spec/amber_lsp/lookup/resolve_api_query_spec.cr index 99a62fd..397a65a 100644 --- a/spec/amber_lsp/lookup/resolve_api_query_spec.cr +++ b/spec/amber_lsp/lookup/resolve_api_query_spec.cr @@ -102,6 +102,15 @@ describe "AmberLSP::Lookup::ResolveAPIQuery#perform" do ]) end + it "keeps a blank documentation return type unknown" do + result = resolve_return_type_fixture_query("Grant::Query::Builder(Post)#untyped") + entry = result.list_of_methods.first + + entry.declared_return_type.should eq("unknown") + entry.resolved_return_type.should eq("unknown") + entry.lookup_signature.should contain(": unknown") + end + it "returns a type summary for a type-only query, including a generic base-name match" do result = resolve_fixture_query("FixtureAPI::Team") generic_result = resolve_fixture_query("Grant::Query::Builder") diff --git a/spec/fixtures/api_lookup/crystal_docs_return_types.json b/spec/fixtures/api_lookup/crystal_docs_return_types.json index 58bfb31..c2c49f7 100644 --- a/spec/fixtures/api_lookup/crystal_docs_return_types.json +++ b/spec/fixtures/api_lookup/crystal_docs_return_types.json @@ -63,6 +63,12 @@ "args_string": "(enabled : Bool) : self", "location": {"filename": "lib/grant/src/grant/query/locking.cr", "line_number": 40}, "def": {"name": "lock", "return_type": "self"} + }, + { + "name": "untyped", + "args_string": "()", + "location": {"filename": "lib/grant/src/grant/query/builder.cr", "line_number": 50}, + "def": {"name": "untyped", "return_type": ""} } ] }, diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr index 2dec431..b9363a5 100644 --- a/src/amber_lsp/lookup/index_cache.cr +++ b/src/amber_lsp/lookup/index_cache.cr @@ -20,7 +20,7 @@ module AmberLSP::Lookup def perform : String root_path = File.expand_path(@root_path) list_of_index_paths = source_paths(root_path) - key_material = ["project", "docs-entries-v1", "resolved-return-types-v1", @compiler_version, @docs_flags.sort.join("\0")] + key_material = ["project", "docs-entries-v1", "resolved-return-types-v2", @compiler_version, @docs_flags.sort.join("\0")] key_material.concat(@docs_entries) list_of_index_paths.each do |path| @@ -61,7 +61,7 @@ module AmberLSP::Lookup key_material = [ "library", "docs-workspace-multi-entry-v2", - "resolved-return-types-v1", + "resolved-return-types-v2", @library_name, @locked_version_or_commit, @docs_flags.sort.join("\0"), @@ -77,7 +77,7 @@ module AmberLSP::Lookup end def perform : String - key_material = ["stdlib", "resolved-return-types-v1", @compiler_version, @docs_flags.sort.join("\0")] + key_material = ["stdlib", "resolved-return-types-v2", @compiler_version, @docs_flags.sort.join("\0")] Digest::SHA256.hexdigest(key_material.join("\0")) end end diff --git a/src/amber_lsp/lookup/index_models.cr b/src/amber_lsp/lookup/index_models.cr index cf3a121..2c718d0 100644 --- a/src/amber_lsp/lookup/index_models.cr +++ b/src/amber_lsp/lookup/index_models.cr @@ -1,6 +1,9 @@ require "json" module AmberLSP::Lookup + # :nodoc: + UNKNOWN_API_RETURN_TYPE = "unknown" + # :nodoc: struct CrystalDocsLocation include JSON::Serializable diff --git a/src/amber_lsp/lookup/normalize_crystal_docs.cr b/src/amber_lsp/lookup/normalize_crystal_docs.cr index 07279cc..e374724 100644 --- a/src/amber_lsp/lookup/normalize_crystal_docs.cr +++ b/src/amber_lsp/lookup/normalize_crystal_docs.cr @@ -72,13 +72,18 @@ module AmberLSP::Lookup doc_line = first_doc_line(docs_method.doc) definition = docs_method.definition return_type = definition ? definition.return_type : nil + declared_return_type = if return_type && !return_type.strip.empty? + return_type + else + UNKNOWN_API_RETURN_TYPE + end APIIndexMethod.new( docs_type.full_name, docs_method.name, method_kind, method_arguments(docs_method.args_string), - return_type || "unknown", + declared_return_type, doc_line, source_path(location.filename), location.line_number, diff --git a/src/amber_lsp/lookup/resolve_api_return_type.cr b/src/amber_lsp/lookup/resolve_api_return_type.cr index d6b5d68..f7cf877 100644 --- a/src/amber_lsp/lookup/resolve_api_return_type.cr +++ b/src/amber_lsp/lookup/resolve_api_return_type.cr @@ -10,7 +10,7 @@ module AmberLSP::Lookup end def perform : String - return "unknown" if @declared_return_type == "unknown" + return UNKNOWN_API_RETURN_TYPE if @declared_return_type == UNKNOWN_API_RETURN_TYPE resolved_type = @declared_return_type if receiver_type = @receiver_type From 6a70f62482d45288b8a8f82aa24177efd83e2f3a Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 01:59:42 -0400 Subject: [PATCH 45/69] fix(lsp): index Crystal docs constructors --- spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr | 8 ++++++++ spec/fixtures/api_lookup/crystal_docs_small.json | 9 +++++++++ src/amber_lsp/lookup/index_models.cr | 2 ++ src/amber_lsp/lookup/normalize_crystal_docs.cr | 6 ++++-- 4 files changed, 23 insertions(+), 2 deletions(-) diff --git a/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr b/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr index 2bf9f14..07eca9a 100644 --- a/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr +++ b/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr @@ -46,5 +46,13 @@ describe "AmberLSP::Lookup::NormalizeCrystalDocs#perform" do where_overloads = query_methods.not_nil!.list_of_instance_methods.select { |method| method.name == "where" } where_overloads.size.should eq(2) where_overloads.map(&.abstract?).should eq([false, true]) + + constructors = user.list_of_class_methods.select { |method| method.name == "new" } + constructors.size.should eq(1) + constructors.first.args_string.should eq("(name : String)") + constructors.first.declared_return_type.should eq("FixtureAPI::User") + constructors.first.resolved_return_type.should eq("FixtureAPI::User") + constructors.first.source_path.should eq("/project/src/models/user.cr") + constructors.first.source_line.should eq(5) end end diff --git a/spec/fixtures/api_lookup/crystal_docs_small.json b/spec/fixtures/api_lookup/crystal_docs_small.json index 7da2b52..b5ea8bf 100644 --- a/spec/fixtures/api_lookup/crystal_docs_small.json +++ b/spec/fixtures/api_lookup/crystal_docs_small.json @@ -35,6 +35,15 @@ "def": {"name": "count", "return_type": "Int32"} } ], + "constructors": [ + { + "name": "new", + "abstract": false, + "args_string": "(name : String)", + "location": {"filename": "src/models/user.cr", "line_number": 5}, + "def": {"name": "new"} + } + ], "macros": [ { "name": "has_token", diff --git a/src/amber_lsp/lookup/index_models.cr b/src/amber_lsp/lookup/index_models.cr index 2c718d0..d57e5e3 100644 --- a/src/amber_lsp/lookup/index_models.cr +++ b/src/amber_lsp/lookup/index_models.cr @@ -66,6 +66,7 @@ module AmberLSP::Lookup getter instance_methods : Array(CrystalDocsMethod) = [] of CrystalDocsMethod @[JSON::Field(key: "class_methods")] getter class_methods : Array(CrystalDocsMethod) = [] of CrystalDocsMethod + getter constructors : Array(CrystalDocsMethod) = [] of CrystalDocsMethod getter macros : Array(CrystalDocsMethod) = [] of CrystalDocsMethod getter types : Array(CrystalDocsType) = [] of CrystalDocsType @@ -80,6 +81,7 @@ module AmberLSP::Lookup @extended_modules : Array(CrystalDocsTypeReference) = [] of CrystalDocsTypeReference, @instance_methods : Array(CrystalDocsMethod) = [] of CrystalDocsMethod, @class_methods : Array(CrystalDocsMethod) = [] of CrystalDocsMethod, + @constructors : Array(CrystalDocsMethod) = [] of CrystalDocsMethod, @macros : Array(CrystalDocsMethod) = [] of CrystalDocsMethod, @types : Array(CrystalDocsType) = [] of CrystalDocsType, ) diff --git a/src/amber_lsp/lookup/normalize_crystal_docs.cr b/src/amber_lsp/lookup/normalize_crystal_docs.cr index e374724..315dff5 100644 --- a/src/amber_lsp/lookup/normalize_crystal_docs.cr +++ b/src/amber_lsp/lookup/normalize_crystal_docs.cr @@ -54,7 +54,8 @@ module AmberLSP::Lookup location_line, @layer_name, normalize_methods(docs_type, docs_type.instance_methods, "instance", false), - normalize_methods(docs_type, docs_type.class_methods, "class", false), + normalize_methods(docs_type, docs_type.class_methods, "class", false) + + normalize_methods(docs_type, docs_type.constructors, "class", false, docs_type.full_name), normalize_methods(docs_type, docs_type.macros, "macro", true), ) end @@ -64,6 +65,7 @@ module AmberLSP::Lookup docs_methods : Array(CrystalDocsMethod), method_kind : String, is_macro : Bool, + return_type_override : String? = nil, ) : Array(APIIndexMethod) docs_methods.compact_map do |docs_method| location = docs_method.location @@ -71,7 +73,7 @@ module AmberLSP::Lookup doc_line = first_doc_line(docs_method.doc) definition = docs_method.definition - return_type = definition ? definition.return_type : nil + return_type = return_type_override || (definition ? definition.return_type : nil) declared_return_type = if return_type && !return_type.strip.empty? return_type else From aba94c8488e772aaa66c4fa198c24c9481518c8d Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 02:04:44 -0400 Subject: [PATCH 46/69] fix(lsp): resolve included module methods --- .../lookup/normalize_crystal_docs_spec.cr | 20 +++++ .../lookup/resolve_api_query_spec.cr | 27 ++++++ .../crystal_docs_included_modules.json | 45 ++++++++++ .../forwarded_collection/crystal_docs.json | 17 ++++ .../src/grant/collection.cr | 7 ++ .../lookup/normalize_crystal_docs.cr | 25 +++++- src/amber_lsp/lookup/resolve_api_query.cr | 84 +++++++++++++++++++ .../lookup/resolve_api_return_type.cr | 76 +++++++++++++---- 8 files changed, 285 insertions(+), 16 deletions(-) create mode 100644 spec/fixtures/api_lookup/crystal_docs_included_modules.json create mode 100644 spec/fixtures/api_lookup/forwarded_collection/crystal_docs.json create mode 100644 spec/fixtures/api_lookup/forwarded_collection/src/grant/collection.cr diff --git a/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr b/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr index 07eca9a..f5e4d68 100644 --- a/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr +++ b/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr @@ -55,4 +55,24 @@ describe "AmberLSP::Lookup::NormalizeCrystalDocs#perform" do constructors.first.source_path.should eq("/project/src/models/user.cr") constructors.first.source_line.should eq(5) end + + it "indexes Enumerable methods forwarded by Grant::Collection" do + fixture_root = File.join(Dir.current, "spec", "fixtures", "api_lookup", "forwarded_collection") + fixture_path = File.join(fixture_root, "crystal_docs.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + fixture_root, + "library", + "grant", + "grant-key", + [] of String, + ).perform + + collection = layer.list_of_types.find { |type| type.name == "Grant::Collection(M)" } + if collection_type = collection + collection_type.list_of_included_module_names.should contain("Enumerable(M)") + else + raise Exception.new("Expected the normalized Grant::Collection type") + end + end end diff --git a/spec/amber_lsp/lookup/resolve_api_query_spec.cr b/spec/amber_lsp/lookup/resolve_api_query_spec.cr index 397a65a..7b85479 100644 --- a/spec/amber_lsp/lookup/resolve_api_query_spec.cr +++ b/spec/amber_lsp/lookup/resolve_api_query_spec.cr @@ -89,6 +89,20 @@ describe "AmberLSP::Lookup::ResolveAPIQuery#perform" do ) end + it "searches generic included modules on a type and its ancestors" do + direct_result = resolve_included_module_fixture_query("Grant::Collection(Post)#to_a") + target_result = resolve_included_module_fixture_query("Grant::AssociationCollection(User, Post)#to_a") + ancestor_result = resolve_included_module_fixture_query("Grant::AncestorCollection(Post)#to_a") + + direct_result.list_of_methods.size.should eq(1) + direct_result.list_of_methods.first.owner.should eq("Enumerable(T)") + direct_result.list_of_methods.first.resolved_return_type.should eq("Array(Post)") + target_result.list_of_methods.first.resolved_return_type.should eq("Array(Post)") + ancestor_result.list_of_methods.size.should eq(1) + ancestor_result.list_of_methods.first.owner.should eq("Enumerable(T)") + ancestor_result.list_of_methods.first.resolved_return_type.should eq("Array(Post)") + end + it "resolves each overload from that row's declared return type" do resolution = resolve_return_type_fixture_query("Grant::Query::Builder(Post)#first") entries = AmberLSP::Lookup::ResolveReturnTypesForAPIIndexMethods.new( @@ -164,3 +178,16 @@ private def resolve_return_type_fixture_query(query : String) : AmberLSP::Lookup ).perform AmberLSP::Lookup::ResolveAPIQuery.new(query, [layer]).perform end + +private def resolve_included_module_fixture_query(query : String) : AmberLSP::Lookup::APIResolution + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_included_modules.json") + layer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + "/project", + "project", + "fixture_included_modules", + "fixture-key", + [] of String, + ).perform + AmberLSP::Lookup::ResolveAPIQuery.new(query, [layer]).perform +end diff --git a/spec/fixtures/api_lookup/crystal_docs_included_modules.json b/spec/fixtures/api_lookup/crystal_docs_included_modules.json new file mode 100644 index 0000000..d989f29 --- /dev/null +++ b/spec/fixtures/api_lookup/crystal_docs_included_modules.json @@ -0,0 +1,45 @@ +{ + "repository_name": "fixture_included_modules", + "program": { + "kind": "module", + "name": "Grant", + "full_name": "Grant", + "types": [ + { + "kind": "class", + "name": "Collection", + "full_name": "Grant::Collection(T)", + "locations": [{"filename": "lib/grant/src/grant/collection.cr", "line_number": 1}], + "included_modules": [{"kind": "module", "name": "Enumerable", "full_name": "Enumerable(T)"}] + }, + { + "kind": "class", + "name": "AncestorCollection", + "full_name": "Grant::AncestorCollection(T)", + "locations": [{"filename": "lib/grant/src/grant/ancestor_collection.cr", "line_number": 1}], + "ancestors": [{"kind": "class", "name": "Collection", "full_name": "Grant::Collection(T)"}] + }, + { + "kind": "class", + "name": "AssociationCollection", + "full_name": "Grant::AssociationCollection(Owner, Target)", + "locations": [{"filename": "lib/grant/src/grant/association_collection.cr", "line_number": 1}], + "included_modules": [{"kind": "module", "name": "Enumerable", "full_name": "Enumerable(Target)"}] + }, + { + "kind": "module", + "name": "Enumerable", + "full_name": "Enumerable(T)", + "locations": [{"filename": "lib/crystal/enumerable.cr", "line_number": 1}], + "instance_methods": [ + { + "name": "to_a", + "args_string": "() : Array(T)", + "location": {"filename": "lib/crystal/enumerable.cr", "line_number": 5}, + "def": {"name": "to_a", "return_type": "Array(T)"} + } + ] + } + ] + } +} diff --git a/spec/fixtures/api_lookup/forwarded_collection/crystal_docs.json b/spec/fixtures/api_lookup/forwarded_collection/crystal_docs.json new file mode 100644 index 0000000..897e238 --- /dev/null +++ b/spec/fixtures/api_lookup/forwarded_collection/crystal_docs.json @@ -0,0 +1,17 @@ +{ + "repository_name": "grant", + "program": { + "kind": "module", + "name": "Grant", + "full_name": "Grant", + "types": [ + { + "kind": "class", + "name": "Collection", + "full_name": "Grant::Collection(M)", + "locations": [{"filename": "src/grant/collection.cr", "line_number": 1}], + "instance_methods": [] + } + ] + } +} diff --git a/spec/fixtures/api_lookup/forwarded_collection/src/grant/collection.cr b/spec/fixtures/api_lookup/forwarded_collection/src/grant/collection.cr new file mode 100644 index 0000000..6edb8cf --- /dev/null +++ b/spec/fixtures/api_lookup/forwarded_collection/src/grant/collection.cr @@ -0,0 +1,7 @@ +class Grant::Collection(M) + forward_missing_to collection + + private def collection : Array(M) + [] of M + end +end diff --git a/src/amber_lsp/lookup/normalize_crystal_docs.cr b/src/amber_lsp/lookup/normalize_crystal_docs.cr index 315dff5..0bdc465 100644 --- a/src/amber_lsp/lookup/normalize_crystal_docs.cr +++ b/src/amber_lsp/lookup/normalize_crystal_docs.cr @@ -1,4 +1,5 @@ require "./index_models" +require "./parse_api_type_name" module AmberLSP::Lookup class NormalizeCrystalDocs @@ -48,7 +49,7 @@ module AmberLSP::Lookup docs_type.kind, docs_type.abstract?, docs_type.ancestors.map(&.full_name), - docs_type.included_modules.map(&.full_name), + included_module_names(docs_type, location_path), docs_type.extended_modules.map(&.full_name), location_path, location_line, @@ -96,6 +97,28 @@ module AmberLSP::Lookup end end + private def included_module_names(docs_type : CrystalDocsType, location_path : String) : Array(String) + module_names = docs_type.included_modules.map(&.full_name) + type_name = ParseAPITypeName.new(docs_type.full_name).perform + return module_names unless type_name.base_name == "Grant::Collection" + return module_names unless source_forwards_collection_to_array?(location_path) + + type_parameter = type_name.list_of_type_arguments.first? + # Crystal docs omits macro-forwarded methods; Grant::Collection forwards its Array(M) surface. + module_names << "Enumerable(#{type_parameter})" if type_parameter + module_names.uniq + end + + private def source_forwards_collection_to_array?(source_path : String) : Bool + return false if source_path.empty? || !File.file?(source_path) + + File.read(source_path).lines.any? do |line| + line.matches?(/\A\s*forward_missing_to\s+collection\b/) + end + rescue ex : IO::Error + false + end + private def method_arguments(args_string : String?) : String return "()" unless args_string diff --git a/src/amber_lsp/lookup/resolve_api_query.cr b/src/amber_lsp/lookup/resolve_api_query.cr index 4191097..da5037b 100644 --- a/src/amber_lsp/lookup/resolve_api_query.cr +++ b/src/amber_lsp/lookup/resolve_api_query.cr @@ -63,6 +63,13 @@ module AmberLSP::Lookup return found_result(query, "instance_method", methods, receiver_type(type_name, matching_types)) unless methods.empty? end + included_methods = included_module_instance_methods( + matching_types, + receiver_type(type_name, matching_types), + method_name, + ) + return APIResolution.new(query, "instance_method", nil, included_methods) unless included_methods.empty? + unknown_result(query) end @@ -127,6 +134,75 @@ module AmberLSP::Lookup unique_types(list_of_ancestors) end + private def included_module_instance_methods( + types : Array(APIIndexType), + receiver_type_name : String, + method_name : String, + ) : Array(APIIndexMethod) + methods = [] of APIIndexMethod + api_type_instances_for(types, receiver_type_name).each do |owner_instance| + owner_bindings = APITypeParameterBindings.new( + owner_instance.type.name, + owner_instance.instantiated_name, + ) + owner_instance.type.list_of_included_module_names.each do |module_template_name| + module_instance_name = owner_bindings.substitute(module_template_name) + types_named(module_instance_name).each do |module_type| + module_bindings = APITypeParameterBindings.new( + module_type.name, + module_instance_name, + ).perform + methods_named([module_type], method_name, "instance").each do |method| + methods << resolve_included_module_method(method, receiver_type_name, module_bindings) + end + end + end + end + unique_methods(methods) + end + + private def resolve_included_module_method( + method : APIIndexMethod, + receiver_type_name : String, + module_bindings : Hash(String, String), + ) : APIIndexMethod + resolved_return_type = ResolveAPIIndexReturnType.new( + method.declared_return_type, + receiver_type_name, + method.owner, + module_bindings, + ).perform + method.with_resolved_return_type(resolved_return_type) + end + + private def api_type_instances_for( + types : Array(APIIndexType), + receiver_type_name : String, + ) : Array(APITypeInstance) + pending_instances = types.map { |type| APITypeInstance.new(type, receiver_type_name) } + list_of_instances = [] of APITypeInstance + seen_instances = Set(String).new + next_instance_index = 0 + + while instance = pending_instances[next_instance_index]? + next_instance_index += 1 + instance_key = "#{instance.type.name}\0#{instance.instantiated_name}" + next if seen_instances.includes?(instance_key) + + seen_instances.add(instance_key) + list_of_instances << instance + owner_bindings = APITypeParameterBindings.new(instance.type.name, instance.instantiated_name) + instance.type.list_of_ancestor_names.each do |ancestor_template_name| + ancestor_instance_name = owner_bindings.substitute(ancestor_template_name) + types_named(ancestor_instance_name).each do |ancestor_type| + pending_instances << APITypeInstance.new(ancestor_type, ancestor_instance_name) + end + end + end + + list_of_instances + end + private def methods_named(types : Array(APIIndexType), method_name : String, method_kind : String) : Array(APIIndexMethod) methods = [] of APIIndexMethod types.each do |type| @@ -215,6 +291,14 @@ module AmberLSP::Lookup APIResolution.new(query, resolution_kind, nil, resolved_methods) end + private struct APITypeInstance + getter type : APIIndexType + getter instantiated_name : String + + def initialize(@type : APIIndexType, @instantiated_name : String) + end + end + private def unknown_result(query : String) : APIResolution APIResolution.new(query, "unknown") end diff --git a/src/amber_lsp/lookup/resolve_api_return_type.cr b/src/amber_lsp/lookup/resolve_api_return_type.cr index f7cf877..aeafc04 100644 --- a/src/amber_lsp/lookup/resolve_api_return_type.cr +++ b/src/amber_lsp/lookup/resolve_api_return_type.cr @@ -1,11 +1,66 @@ require "./parse_api_type_name" module AmberLSP::Lookup + class APITypeParameterBindings + def initialize(@type_template_name : String, @instantiated_type_name : String) + end + + def perform : Hash(String, String) + template_type = ParseAPITypeName.new(@type_template_name).perform + instantiated_type = ParseAPITypeName.new(@instantiated_type_name).perform + return {} of String => String unless template_type.base_name == instantiated_type.base_name + return {} of String => String unless template_type.list_of_type_arguments.size == instantiated_type.list_of_type_arguments.size + + bindings = {} of String => String + template_type.list_of_type_arguments.each_with_index do |parameter_name, index| + argument = instantiated_type.list_of_type_arguments[index]? + bindings[parameter_name] = argument if argument + end + bindings + end + + def substitute(type_expression : String) : String + bindings = perform + return type_expression if bindings.empty? + + output = String::Builder.new + current_token = String::Builder.new + type_expression.each_char do |character| + if type_token_character?(character) + current_token << character + else + append_substituted_token(output, current_token, bindings) + current_token = String::Builder.new + output << character + end + end + append_substituted_token(output, current_token, bindings) + output.to_s + end + + private def type_token_character?(character : Char) : Bool + (character >= 'A' && character <= 'Z') || + (character >= 'a' && character <= 'z') || + (character >= '0' && character <= '9') || + character == '_' || character == ':' + end + + private def append_substituted_token( + output : String::Builder, + current_token : String::Builder, + bindings : Hash(String, String), + ) : Nil + token = current_token.to_s + output << (bindings[token]? || token) + end + end + class ResolveAPIIndexReturnType def initialize( @declared_return_type : String, @receiver_type : String?, @owner_type_name : String? = nil, + @additional_type_parameter_bindings : Hash(String, String) = {} of String => String, ) end @@ -27,23 +82,14 @@ module AmberLSP::Lookup private def substitute_owner_type_parameters(return_type : String) : String owner_type_name = @owner_type_name receiver_type = @receiver_type - return return_type unless owner_type_name && receiver_type - - owner_type = ParseAPITypeName.new(owner_type_name).perform - receiver = ParseAPITypeName.new(receiver_type).perform - return return_type unless normalize_type_base(owner_type.base_name) == normalize_type_base(receiver.base_name) - return return_type unless owner_type.list_of_type_arguments.size == receiver.list_of_type_arguments.size - - bindings = {} of String => String - owner_type.list_of_type_arguments.each_with_index do |parameter_name, index| - argument = receiver.list_of_type_arguments[index]? - bindings[parameter_name] = argument if argument + bindings = @additional_type_parameter_bindings.dup + if owner_type_name && receiver_type + owner_bindings = APITypeParameterBindings.new(owner_type_name, receiver_type).perform + bindings.merge!(owner_bindings) end - substitute_type_tokens(return_type, bindings) - end + return return_type if bindings.empty? - private def normalize_type_base(name : String) : String - name.sub(/\A::/, "") + substitute_type_tokens(return_type, bindings) end private def substitute_type_tokens(return_type : String, bindings : Hash(String, String)) : String From 61e38c8f5e66290fdd13d131df36ece805b3f409 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 02:15:04 -0400 Subject: [PATCH 47/69] feat(lsp): verify lookup return types --- spec/amber_lsp/lookup/lookup_command_spec.cr | 9 + .../amber_lsp/lookup/verify_api_query_spec.cr | 111 ++++++- src/amber_lsp/lookup/index_models.cr | 48 +++ src/amber_lsp/lookup/lookup_command.cr | 7 + src/amber_lsp/lookup/verify_api_query.cr | 287 +++++++++++++++++- 5 files changed, 451 insertions(+), 11 deletions(-) diff --git a/spec/amber_lsp/lookup/lookup_command_spec.cr b/spec/amber_lsp/lookup/lookup_command_spec.cr index df00343..faee639 100644 --- a/spec/amber_lsp/lookup/lookup_command_spec.cr +++ b/spec/amber_lsp/lookup/lookup_command_spec.cr @@ -120,9 +120,18 @@ describe AmberLSP::Lookup::RunLookupCommand do json["entries"].as_a.first["name"].as_s.should eq("id") json["entries"].as_a.first["declared_return_type"].as_s.should eq("Int64 | ::Nil") json["entries"].as_a.first["resolved_return_type"].as_s.should eq("Int64 | Nil") + json["entries"].as_a.first["verified_return_type"].as_s.should eq("Int64 | Nil") + json["verified_return_type"].as_s.should eq("Int64 | Nil") json["verification_status"].as_s.should eq("present") json["verification_elapsed_milliseconds"].as_i.should be >= 0 json["card_notes"].as_a.size.should eq(1) + + text_result = run_lookup_cli( + ["CLIProject::User#id", "--root", project, "--verify"], + lookup_cli_cache_root, + ) + text_result[0].should eq(0) + text_result[1].should contain("verified type: Int64 | Nil") end end diff --git a/spec/amber_lsp/lookup/verify_api_query_spec.cr b/spec/amber_lsp/lookup/verify_api_query_spec.cr index d14708d..67b805a 100644 --- a/spec/amber_lsp/lookup/verify_api_query_spec.cr +++ b/spec/amber_lsp/lookup/verify_api_query_spec.cr @@ -44,6 +44,83 @@ describe AmberLSP::Lookup::VerifyAPIQuery do end end + it "reports the compiler return type for an indexed overload with an unknown docs type" do + with_probe_project do |project| + entry = probe_method("ProbeAPI::Child", "select", "class", "()", "unknown") + resolution = AmberLSP::Lookup::APIResolution.new( + "ProbeAPI::Child.select", + "class_method", + nil, + [entry], + ) + + result = AmberLSP::Lookup::VerifyAPIQuery.new(project, resolution.query, resolution).perform + + result.status.should eq("present") + result.output.should contain("verified type: String") + end + end + + it "reports a reason when an overload cannot be type checked because it has a splat" do + with_probe_project do |project| + entry = probe_method("ProbeAPI::Child", "choices", "class", "(*fields : String)", "unknown") + resolution = AmberLSP::Lookup::APIResolution.new( + "ProbeAPI::Child.choices", + "class_method", + nil, + [entry], + ) + + result = AmberLSP::Lookup::VerifyAPIQuery.new(project, resolution.query, resolution).perform + + result.output.should contain("type check skipped: splat parameters are unsupported") + result.status.should eq("present") + end + end + + it "builds a typed call from declared parameter types" do + with_probe_project do |project| + entry = probe_method("ProbeAPI::Child", "convert", "instance", "(value : String)", "unknown") + resolution = AmberLSP::Lookup::APIResolution.new( + "ProbeAPI::Child#convert", + "instance_method", + nil, + [entry], + ) + + result = AmberLSP::Lookup::VerifyAPIQuery.new(project, resolution.query, resolution).perform + + result.status.should eq("present") + result.verified_return_type.should eq("String") + result.list_of_verified_entries.first.verified_return_type.should eq("String") + end + end + + it "skips block and untyped parameters with a reason" do + with_probe_project do |project| + untyped_entry = probe_method("ProbeAPI::Child", "untyped_parameter", "class", "(value)", "unknown") + untyped_resolution = AmberLSP::Lookup::APIResolution.new( + "ProbeAPI::Child.untyped_parameter", + "class_method", + nil, + [untyped_entry], + ) + block_entry = probe_method("ProbeAPI::Child", "transform", "class", "(&block : String -> Bool)", "unknown") + block_resolution = AmberLSP::Lookup::APIResolution.new( + "ProbeAPI::Child.transform", + "class_method", + nil, + [block_entry], + ) + + untyped_result = AmberLSP::Lookup::VerifyAPIQuery.new(project, untyped_resolution.query, untyped_resolution).perform + block_result = AmberLSP::Lookup::VerifyAPIQuery.new(project, block_resolution.query, block_resolution).perform + + untyped_result.output.should contain("type check skipped: parameters without declared types are unsupported") + block_result.output.should contain("type check skipped: block parameters are unsupported") + end + end + it "returns unavailable when the query cannot be represented safely" do with_probe_project do |project| result = verify_query(project, "ProbeAPI::Child#safe_name; puts 1") @@ -58,13 +135,19 @@ private def verify_query(project : String, query : String) : AmberLSP::Lookup::A AmberLSP::Lookup::VerifyAPIQuery.new(project, query, resolution).perform end -private def probe_method(owner : String, name : String) : AmberLSP::Lookup::APIIndexMethod +private def probe_method( + owner : String, + name : String, + method_kind : String = "instance", + args_string : String = "", + declared_return_type : String = "", +) : AmberLSP::Lookup::APIIndexMethod AmberLSP::Lookup::APIIndexMethod.new( owner, name, - "instance", - "", - "", + method_kind, + args_string, + declared_return_type, nil, "src/probe_api.cr", 1, @@ -106,6 +189,26 @@ private def with_probe_project(&) class Child < Parent extend ClassExtensions + + def self.select : String + "selected" + end + + def self.choices(*fields : String) : Array(String) + fields.to_a + end + + def self.untyped_parameter(value) + value.to_s + end + + def self.transform(&block : String -> Bool) : String + block.call("value") ? "yes" : "no" + end + + def convert(value : String) : String + value + end end class FirstCandidate diff --git a/src/amber_lsp/lookup/index_models.cr b/src/amber_lsp/lookup/index_models.cr index d57e5e3..018e48e 100644 --- a/src/amber_lsp/lookup/index_models.cr +++ b/src/amber_lsp/lookup/index_models.cr @@ -118,6 +118,10 @@ module AmberLSP::Lookup getter source_layer : String getter? abstract : Bool getter? macro : Bool + @[JSON::Field(emit_null: true)] + getter verified_return_type : String? = nil + @[JSON::Field(emit_null: true)] + getter verification_skip_reason : String? = nil def initialize( @owner : String, @@ -132,6 +136,8 @@ module AmberLSP::Lookup @abstract : Bool, @macro : Bool, resolved_return_type : String? = nil, + @verified_return_type : String? = nil, + @verification_skip_reason : String? = nil, ) @resolved_return_type = resolved_return_type || @declared_return_type end @@ -154,6 +160,8 @@ module AmberLSP::Lookup abstract?, macro?, resolved_return_type, + verified_return_type, + verification_skip_reason, ) end @@ -171,6 +179,46 @@ module AmberLSP::Lookup abstract?, macro?, resolved_type, + verified_return_type, + verification_skip_reason, + ) + end + + def with_verified_return_type(verified_type : String) : APIIndexMethod + APIIndexMethod.new( + owner, + name, + method_kind, + args_string, + declared_return_type, + doc_line, + source_path, + source_line, + source_layer, + abstract?, + macro?, + resolved_return_type, + verified_type, + nil, + ) + end + + def with_verification_skip_reason(reason : String) : APIIndexMethod + APIIndexMethod.new( + owner, + name, + method_kind, + args_string, + declared_return_type, + doc_line, + source_path, + source_line, + source_layer, + abstract?, + macro?, + resolved_return_type, + nil, + reason, ) end diff --git a/src/amber_lsp/lookup/lookup_command.cr b/src/amber_lsp/lookup/lookup_command.cr index 69b95be..138bbb1 100644 --- a/src/amber_lsp/lookup/lookup_command.cr +++ b/src/amber_lsp/lookup/lookup_command.cr @@ -42,6 +42,8 @@ module AmberLSP::Lookup @[JSON::Field(emit_null: true)] getter verification_elapsed_milliseconds : Int64? @[JSON::Field(emit_null: true)] + getter verified_return_type : String? + @[JSON::Field(emit_null: true)] getter failure_reason : String? @[JSON::Field(key: "card_notes")] getter list_of_card_notes : Array(APICardNote) @@ -59,6 +61,7 @@ module AmberLSP::Lookup @list_of_layers : Array(APIIndexLayerState), @verification_status : String?, @verification_elapsed_milliseconds : Int64?, + @verified_return_type : String?, @failure_reason : String?, @list_of_card_notes : Array(APICardNote) = [] of APICardNote, @list_of_error_hints : Array(APICardErrorHint) = [] of APICardErrorHint, @@ -197,6 +200,7 @@ module AmberLSP::Lookup when "unavailable" failure_reason = probe.output end + list_of_entries = probe.list_of_verified_entries unless probe.list_of_verified_entries.empty? end LookupCLIAnswer.new( @@ -208,6 +212,7 @@ module AmberLSP::Lookup answer.list_of_layers, verification_status, probe.try(&.elapsed_milliseconds), + probe.try(&.verified_return_type), failure_reason, list_of_card_notes, list_of_error_hints, @@ -224,6 +229,8 @@ module AmberLSP::Lookup answer.list_of_entries.each do |entry| @stdout.puts("#{entry.lookup_signature} — #{entry.source_path}:#{entry.source_line} [#{entry.source_layer}]") @stdout.puts(" #{entry.doc_line}") if entry.doc_line + @stdout.puts(" verified type: #{entry.verified_return_type}") if entry.verified_return_type + @stdout.puts(" type check skipped: #{entry.verification_skip_reason}") if entry.verification_skip_reason end answer.list_of_card_notes.each do |note| diff --git a/src/amber_lsp/lookup/verify_api_query.cr b/src/amber_lsp/lookup/verify_api_query.cr index 372db06..6efd190 100644 --- a/src/amber_lsp/lookup/verify_api_query.cr +++ b/src/amber_lsp/lookup/verify_api_query.cr @@ -16,12 +16,16 @@ module AmberLSP::Lookup getter elapsed_milliseconds : Int64 getter output : String getter verified_entry : APIIndexMethod? + getter list_of_verified_entries : Array(APIIndexMethod) + getter verified_return_type : String? def initialize( @status : String, @elapsed_milliseconds : Int64, @output : String, @verified_entry : APIIndexMethod? = nil, + @list_of_verified_entries : Array(APIIndexMethod) = [] of APIIndexMethod, + @verified_return_type : String? = nil, ) end end @@ -55,18 +59,22 @@ module AmberLSP::Lookup private def verification_target : VerificationTarget? if match = @query.strip.match(/\A(.+)#([^.#]+)\z/) - return VerificationTarget.new(match[1], match[2], "instance", nil) if safe_target?(match[1], match[2]) + if safe_target?(match[1], match[2]) + return VerificationTarget.new(match[1], match[2], "instance", nil, entries_named(match[2])) + end return nil end if match = @query.strip.match(/\A(.+)\.([^.#]+)\z/) - return VerificationTarget.new(match[1], match[2], "class", nil) if safe_target?(match[1], match[2]) + if safe_target?(match[1], match[2]) + return VerificationTarget.new(match[1], match[2], "class", nil, entries_named(match[2])) + end return nil end if @query.strip[0]?.try(&.uppercase?) type_name = @query.strip - return VerificationTarget.new(type_name, nil, "type", nil) if safe_type_name?(type_name) + return VerificationTarget.new(type_name, nil, "type", nil, [] of APIIndexMethod) if safe_type_name?(type_name) return nil end @@ -81,15 +89,20 @@ module AmberLSP::Lookup next if entry.method_kind == "macro" next unless safe_target?(entry.owner, entry.name) - target_key = [entry.owner, entry.name, entry.method_kind].join("\0") + target_kind = entry.method_kind == "instance" ? "instance" : "class" + target_key = [entry.owner, entry.name, target_kind].join("\0") next if seen_targets.includes?(target_key) seen_targets.add(target_key) - targets << VerificationTarget.new(entry.owner, entry.name, entry.method_kind, entry) + targets << VerificationTarget.new(entry.owner, entry.name, target_kind, entry, [entry]) end targets end + private def entries_named(method_name : String) : Array(APIIndexMethod) + @resolution.list_of_methods.select { |entry| entry.name == method_name } + end + private def verify_candidates(started_at : Time::Instant, compiler_path : String) : APIProbeResult targets = candidate_targets return unavailable(started_at, "The index returned no safe candidates for this query") if targets.empty? @@ -130,7 +143,24 @@ module AmberLSP::Lookup compiler_output = [output.to_s, error_output.to_s].reject(&.empty?).join("\n").strip if status.success? - APIProbeResult.new("present", elapsed_milliseconds(started_at), compiler_output, target.verified_entry) + list_of_verified_entries, return_type_output = verify_return_types( + compiler_path, + probe_root, + cache_path, + require_path, + target, + ) + combined_output = [compiler_output, return_type_output.join("\n")].reject(&.empty?).join("\n") + verified_entry = verified_candidate_entry(target, list_of_verified_entries) + verified_return_type = list_of_verified_entries.compact_map(&.verified_return_type).first? + APIProbeResult.new( + "present", + elapsed_milliseconds(started_at), + combined_output, + verified_entry, + list_of_verified_entries, + verified_return_type, + ) elsif compiler_output.includes?(ABSENT_MARKER) APIProbeResult.new("absent", elapsed_milliseconds(started_at), compiler_output) else @@ -142,6 +172,234 @@ module AmberLSP::Lookup FileUtils.rm_rf(probe_root) if probe_root end + private def verify_return_types( + compiler_path : String, + probe_root : String, + cache_path : String, + require_path : String, + target : VerificationTarget, + ) : Tuple(Array(APIIndexMethod), Array(String)) + list_of_entries = [] of APIIndexMethod + output_lines = [] of String + return {list_of_entries, output_lines} unless target.method_name && !target.list_of_entries.empty? + + target.list_of_entries.each_with_index do |entry, index| + if entry.macro? + reason = "macros do not have a call-time return type" + list_of_entries << entry.with_verification_skip_reason(reason) + output_lines << "type check skipped: #{reason}" + next + end + + parsed_arguments = parse_argument_types(entry, target.type_name) + if skip_reason = parsed_arguments.skip_reason + list_of_entries << entry.with_verification_skip_reason(skip_reason) + output_lines << "type check skipped: #{skip_reason}" + next + end + + call_expression = method_call_expression(target, entry, parsed_arguments.list_of_types.size) + probe_path = File.join(probe_root, "type_probe_#{index}.cr") + File.write( + probe_path, + return_type_probe_source(require_path, target, parsed_arguments.list_of_types, call_expression), + ) + + output = IO::Memory.new + error_output = IO::Memory.new + status = Process.run( + compiler_path, + ["run", probe_path], + chdir: @project_root_path, + env: {"CRYSTAL_CACHE_DIR" => cache_path}, + output: output, + error: error_output, + ) + compiler_output = [output.to_s, error_output.to_s].reject(&.empty?).join("\n").strip + verified_type = status.success? ? verified_type_from(compiler_output) : nil + if verified_type + list_of_entries << entry.with_verified_return_type(verified_type) + output_lines << "verified type: #{verified_type}" + else + reason = if status.success? + "compiler did not report a return type" + else + "compiler could not type-check this overload: #{compiler_output}" + end + list_of_entries << entry.with_verification_skip_reason(reason) + output_lines << "type check skipped: #{reason}" + end + end + + {list_of_entries, output_lines} + end + + private def parse_argument_types(entry : APIIndexMethod, receiver_type_name : String) : ParsedArgumentTypes + args_string = entry.args_string.strip + return ParsedArgumentTypes.new([] of String) if args_string.empty? || args_string == "()" + return ParsedArgumentTypes.new([] of String, "overload arguments could not be parsed") unless args_string.starts_with?('(') && args_string.ends_with?(')') + + argument_text = args_string.byte_slice(1, args_string.bytesize - 2) + return ParsedArgumentTypes.new([] of String) if argument_text.strip.empty? + + list_of_types = [] of String + split_method_arguments(argument_text).each do |argument| + normalized_argument = argument.strip + return ParsedArgumentTypes.new([] of String, "block parameters are unsupported") if normalized_argument.starts_with?('&') + return ParsedArgumentTypes.new([] of String, "splat parameters are unsupported") if normalized_argument.starts_with?('*') + + match = normalized_argument.match(/\A[A-Za-z_][A-Za-z0-9_]*\s*:\s*(.+)\z/) + return ParsedArgumentTypes.new([] of String, "parameters without declared types are unsupported") unless match + + declared_type = remove_default_argument(match[1]).strip + return ParsedArgumentTypes.new([] of String, "parameters without declared types are unsupported") if declared_type.empty? + + resolved_type = ResolveAPIIndexReturnType.new( + declared_type, + receiver_type_name, + entry.owner, + ).perform + return ParsedArgumentTypes.new([] of String, "parameter type could not be resolved") if resolved_type == UNKNOWN_API_RETURN_TYPE + + list_of_types << resolved_type + end + + ParsedArgumentTypes.new(list_of_types) + end + + private def split_method_arguments(arguments : String) : Array(String) + list_of_arguments = [] of String + current_argument = String::Builder.new + delimiter_depth = 0 + quote : Char? = nil + escaped = false + + arguments.each_char do |character| + if active_quote = quote + current_argument << character + if escaped + escaped = false + elsif character == '\\' + escaped = true + elsif character == active_quote + quote = nil + end + next + end + + if character == '"' || character == '\'' + quote = character + current_argument << character + next + end + + case character + when '(', '[', '{' then delimiter_depth += 1 + when ')', ']', '}' then delimiter_depth -= 1 if delimiter_depth > 0 + when ',' + if delimiter_depth == 0 + list_of_arguments << current_argument.to_s.strip + current_argument = String::Builder.new + next + end + end + current_argument << character + end + + final_argument = current_argument.to_s.strip + list_of_arguments << final_argument unless final_argument.empty? + list_of_arguments + end + + private def remove_default_argument(argument : String) : String + delimiter_depth = 0 + argument.each_char_with_index do |character, index| + case character + when '(', '[', '{' then delimiter_depth += 1 + when ')', ']', '}' then delimiter_depth -= 1 if delimiter_depth > 0 + when '=' + return argument.byte_slice(0, index).to_s if delimiter_depth == 0 + end + end + argument + end + + private def method_call_expression( + target : VerificationTarget, + entry : APIIndexMethod, + argument_count : Int32, + ) : String + method_name = target.method_name || entry.name + receiver_expression = if target.target_kind == "instance" + "amber_lsp_receiver" + else + target.type_name + end + arguments = (0...argument_count).map { |index| "amber_lsp_argument_#{index}" }.join(", ") + "#{receiver_expression}.#{method_name}(#{arguments})" + end + + private def return_type_probe_source( + require_path : String, + target : VerificationTarget, + list_of_argument_types : Array(String), + call_expression : String, + ) : String + String.build do |source| + source << "require " << require_path.inspect << "\n\n" + if target.target_kind == "instance" + source << "amber_lsp_receiver = uninitialized " << target.type_name << "\n" + end + list_of_argument_types.each_with_index do |type_name, index| + source << "amber_lsp_argument_" << index.to_s << " = uninitialized (" << type_name << ")\n" + end + source << "puts \"AMBER_LSP_VERIFIED_TYPE: " + source << '#' + source << "{typeof(" + source << call_expression + source << ")}\"\n" + end + end + + private def verified_type_from(compiler_output : String) : String? + match = compiler_output.match(/AMBER_LSP_VERIFIED_TYPE:\s*(.+)/) + return nil unless match + + verified_type = match[1].strip + remove_outer_type_parentheses(verified_type) + end + + private def remove_outer_type_parentheses(type_name : String) : String + return type_name unless type_name.starts_with?('(') && type_name.ends_with?(')') + + depth = 0 + type_name.each_char_with_index do |character, index| + case character + when '(' then depth += 1 + when ')' then depth -= 1 + end + return type_name if depth == 0 && index < type_name.size - 1 + end + + type_name.byte_slice(1, type_name.bytesize - 2).strip + end + + private def verified_candidate_entry( + target : VerificationTarget, + list_of_verified_entries : Array(APIIndexMethod), + ) : APIIndexMethod? + verified_candidate = target.verified_entry + return nil unless verified_candidate + + list_of_verified_entries.find do |entry| + entry.owner == verified_candidate.owner && + entry.name == verified_candidate.name && + entry.args_string == verified_candidate.args_string && + entry.source_path == verified_candidate.source_path && + entry.source_line == verified_candidate.source_line + end || verified_candidate + end + private def safe_target?(type_name : String, method_name : String) : Bool safe_type_name?(type_name) && method_name.matches?(/\A[a-zA-Z_][a-zA-Z0-9_]*[!?]?\z/) end @@ -258,8 +516,23 @@ module AmberLSP::Lookup getter method_name : String? getter target_kind : String getter verified_entry : APIIndexMethod? + getter list_of_entries : Array(APIIndexMethod) + + def initialize( + @type_name : String, + @method_name : String?, + @target_kind : String, + @verified_entry : APIIndexMethod?, + @list_of_entries : Array(APIIndexMethod), + ) + end + end + + private struct ParsedArgumentTypes + getter list_of_types : Array(String) + getter skip_reason : String? - def initialize(@type_name : String, @method_name : String?, @target_kind : String, @verified_entry : APIIndexMethod?) + def initialize(@list_of_types : Array(String), @skip_reason : String? = nil) end end end From bdef75af41a569d679bd84756bb52345723f6935 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 02:16:44 -0400 Subject: [PATCH 48/69] feat(lsp): interpolate API card hint captures --- docs/api-cards.md | 25 ++++++++++++++++ spec/amber_lsp/lookup/api_cards_spec.cr | 27 +++++++++++++++++ spec/amber_lsp/lookup/hint_command_spec.cr | 30 +++++++++++++++++++ .../.amber-lsp/api/fixture_project.yml | 10 +++++++ src/amber_lsp/lookup/api_cards.cr | 29 +++++++++++++++--- 5 files changed, 117 insertions(+), 4 deletions(-) create mode 100644 docs/api-cards.md create mode 100644 spec/fixtures/api_lookup/capture_cards/.amber-lsp/api/fixture_project.yml diff --git a/docs/api-cards.md b/docs/api-cards.md new file mode 100644 index 0000000..a50f328 --- /dev/null +++ b/docs/api-cards.md @@ -0,0 +1,25 @@ +# API cards + +Libraries can publish versioned lookup guidance in `.amber-lsp/api/*.yml`. +Each card declares `card_version`, `library`, and `applies_to`; optional fields +include `docs_flags`, `docs_entries`, `notes`, and `error_hints`. + +An `error_hints` item contains a regular expression, hint text, and a right-form +example. The hint and example can interpolate captured text with `{1}`, `{2}`, +or a named placeholder such as `{method}`. Numbered placeholders use the +corresponding regular-expression capture group, starting at 1. Named +placeholders use named groups such as `(?[a-z_]+)`. A placeholder with +no matching capture remains unchanged. + +```yaml +error_hints: + - pattern: "undefined method '([a-z_]+)' for ([A-Za-z_:]+)" + hint: "Call {1} on {2}." + example: "{2}.{1}()" + - pattern: "undefined method '(?[a-z_]+)' for Example::Widget" + hint: "Try {method} on the widget." + example: "Example::Widget.{method}()" +``` + +The `amber-lsp hint` command prints matching hints with the card's resolved +library version. diff --git a/spec/amber_lsp/lookup/api_cards_spec.cr b/spec/amber_lsp/lookup/api_cards_spec.cr index 13eb2b4..ad083cb 100644 --- a/spec/amber_lsp/lookup/api_cards_spec.cr +++ b/spec/amber_lsp/lookup/api_cards_spec.cr @@ -73,6 +73,24 @@ describe AmberLSP::Lookup::LoadAPICards do invalid_pattern_result.list_of_errors.first.should contain(card_path) end end + + it "substitutes numbered and named captures in matching card hints and examples" do + with_capture_card_project do |project| + cards = AmberLSP::Lookup::LoadAPICards.new(project).perform + matches = cards.matching_error_hint_matches("undefined method 'lookup' for Example::Widget") + + matches.map(&.error_hint.hint).should eq([ + "Call lookup on Example::Widget.", + "Use lookup with the widget.", + ]) + matches.map(&.error_hint.example).should eq([ + "Example::Widget.lookup(\"widget-1\")", + "Example::Widget.lookup(\"widget-2\")", + ]) + matches.map(&.card_library).should eq(["fixture_project", "fixture_project"]) + matches.map(&.card_version).should eq(["1.5.0", "1.5.0"]) + end + end end private def with_api_card_project(&) @@ -87,3 +105,12 @@ private def with_api_card_project(&) yield project end end + +private def with_capture_card_project(&) + with_tempdir do |project| + fixture_root = File.join(Dir.current, "spec", "fixtures", "api_lookup", "capture_cards") + File.write(File.join(project, "shard.yml"), "name: fixture_project\nversion: 1.5.0\n") + FileUtils.cp_r(File.join(fixture_root, ".amber-lsp"), project) + yield project + end +end diff --git a/spec/amber_lsp/lookup/hint_command_spec.cr b/spec/amber_lsp/lookup/hint_command_spec.cr index 456f355..1d07bd7 100644 --- a/spec/amber_lsp/lookup/hint_command_spec.cr +++ b/spec/amber_lsp/lookup/hint_command_spec.cr @@ -25,6 +25,27 @@ describe AmberLSP::Lookup::RunHintCommand do end end + it "prints interpolated numbered and named captures with the card version" do + with_capture_hint_cli_project do |project| + input = IO::Memory.new("undefined method 'lookup' for Example::Widget\n") + output = IO::Memory.new + code = AmberLSP::Lookup::RunHintCommand.new( + ["--root", project], + input, + output, + IO::Memory.new, + ).perform + + code.should eq(0) + output.to_s.lines.should eq([ + "hint: Call lookup on Example::Widget. [fixture_project@1.5.0]", + " right: Example::Widget.lookup(\"widget-1\")", + "hint: Use lookup with the widget. [fixture_project@1.5.0]", + " right: Example::Widget.lookup(\"widget-2\")", + ]) + end + end + it "exits successfully without output when no card hint matches" do with_hint_cli_project do |project| input = IO::Memory.new("undefined method 'unlisted'\n") @@ -76,3 +97,12 @@ private def with_hint_cli_project(&) yield project end end + +private def with_capture_hint_cli_project(&) + with_tempdir do |project| + fixture_root = File.join(Dir.current, "spec", "fixtures", "api_lookup", "capture_cards") + File.write(File.join(project, "shard.yml"), "name: fixture_project\nversion: 1.5.0\n") + FileUtils.cp_r(File.join(fixture_root, ".amber-lsp"), project) + yield project + end +end diff --git a/spec/fixtures/api_lookup/capture_cards/.amber-lsp/api/fixture_project.yml b/spec/fixtures/api_lookup/capture_cards/.amber-lsp/api/fixture_project.yml new file mode 100644 index 0000000..c0ac4fe --- /dev/null +++ b/spec/fixtures/api_lookup/capture_cards/.amber-lsp/api/fixture_project.yml @@ -0,0 +1,10 @@ +card_version: 1 +library: fixture_project +applies_to: ">= 1.0.0, < 2.0.0" +error_hints: + - pattern: "undefined method '([a-z_]+)' for ([A-Za-z_:]+)" + hint: "Call {1} on {2}." + example: "{2}.{1}(\"widget-1\")" + - pattern: "undefined method '(?[a-z_]+)' for Example::Widget" + hint: "Use {method} with the widget." + example: "Example::Widget.{method}(\"widget-2\")" diff --git a/src/amber_lsp/lookup/api_cards.cr b/src/amber_lsp/lookup/api_cards.cr index b689dea..a0f9550 100644 --- a/src/amber_lsp/lookup/api_cards.cr +++ b/src/amber_lsp/lookup/api_cards.cr @@ -20,6 +20,9 @@ module AmberLSP::Lookup getter pattern : String getter hint : String getter example : String + + def initialize(@pattern : String, @hint : String, @example : String) + end end struct APICard @@ -93,13 +96,31 @@ module AmberLSP::Lookup def matching_error_hint_matches(error_text : String) : Array(APICardErrorHintMatch) @list_of_cards.flat_map do |loaded_card| - loaded_card.card.error_hints.select do |hint| - Regex.new(hint.pattern).matches?(error_text) - end.map do |hint| - APICardErrorHintMatch.new(loaded_card.card.library, loaded_card.resolved_version, hint) + loaded_card.card.error_hints.compact_map do |hint| + match = Regex.new(hint.pattern).match(error_text) + next unless match + + interpolated_hint = APICardErrorHint.new( + hint.pattern, + substitute_captures(hint.hint, match), + substitute_captures(hint.example, match), + ) + APICardErrorHintMatch.new(loaded_card.card.library, loaded_card.resolved_version, interpolated_hint) end end end + + private def substitute_captures(template : String, match : Regex::MatchData) : String + template.gsub(/\{([^{}]+)\}/) do |placeholder| + capture_name = placeholder.byte_slice(1, placeholder.bytesize - 2) + capture = if capture_number = capture_name.to_i? + match[capture_number]? + else + match[capture_name]? + end + capture || placeholder + end + end end class APIVersionRequirement From 18a4ba158941bf24703682d409e398517eb04ae2 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 02:19:24 -0400 Subject: [PATCH 49/69] fix(lsp): store portable library index paths --- spec/amber_lsp/lookup/index_cache_spec.cr | 61 +++++++++++ src/amber_lsp/lookup/index_cache.cr | 121 +++++++++++++++++++++- 2 files changed, 178 insertions(+), 4 deletions(-) diff --git a/spec/amber_lsp/lookup/index_cache_spec.cr b/spec/amber_lsp/lookup/index_cache_spec.cr index a7d3781..830676a 100644 --- a/spec/amber_lsp/lookup/index_cache_spec.cr +++ b/spec/amber_lsp/lookup/index_cache_spec.cr @@ -119,4 +119,65 @@ describe "AmberLSP::Lookup::APIIndexCache#write_layer and #load_layer" do other_project.failure_reason.should be_nil end end + + it "stores library source paths relative to the shard and rebases them for the querying project" do + with_tempdir do |root| + cache_root = File.join(root, "shared_cache") + source_project = File.join(root, "source_project") + querying_project = File.join(root, "querying_project") + source_library = File.join(source_project, "lib", "grant") + querying_library = File.join(querying_project, "lib", "grant") + source_path = File.join(source_library, "src", "grant", "columns.cr") + + method_entry = AmberLSP::Lookup::APIIndexMethod.new( + "Grant::Column", + "composite_primary_key?", + "instance", + "", + "Bool", + nil, + source_path, + 904, + "grant", + false, + false, + ) + type_entry = AmberLSP::Lookup::APIIndexType.new( + "Grant::Column", + "class", + false, + [] of String, + [] of String, + [] of String, + source_path, + 1, + "grant", + [method_entry], + [] of AmberLSP::Lookup::APIIndexMethod, + [] of AmberLSP::Lookup::APIIndexMethod, + ) + layer = AmberLSP::Lookup::APIIndexLayer.new( + "library", + "grant", + "grant-key-v1", + source_library, + [] of String, + [type_entry], + ) + + source_cache = AmberLSP::Lookup::APIIndexCache.new(cache_root, source_project) + stored_path = source_cache.write_layer(layer) + stored_layer = AmberLSP::Lookup::APIIndexLayer.from_json(File.read(stored_path)) + querying_cache = AmberLSP::Lookup::APIIndexCache.new(cache_root, querying_project) + loaded = querying_cache.load_layer("library", "grant", "grant-key-v1").layer || + raise ArgumentError.new("Expected the cached Grant layer to load for the querying project") + + stored_layer.root_path.should eq(".") + stored_layer.list_of_types.first.location_path.should eq("src/grant/columns.cr") + stored_layer.list_of_types.first.list_of_instance_methods.first.source_path.should eq("src/grant/columns.cr") + loaded.root_path.should eq(querying_library) + loaded.list_of_types.first.location_path.should eq(File.join(querying_library, "src", "grant", "columns.cr")) + loaded.list_of_types.first.list_of_instance_methods.first.source_path.should eq(File.join(querying_library, "src", "grant", "columns.cr")) + end + end end diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr index b9363a5..7a22eae 100644 --- a/src/amber_lsp/lookup/index_cache.cr +++ b/src/amber_lsp/lookup/index_cache.cr @@ -5,6 +5,8 @@ require "random/secure" require "./index_models" module AmberLSP::Lookup + API_INDEX_CACHE_SCHEMA_VERSION = "api-index-cache-v5" + class APIIndexBuildError < Exception end @@ -20,7 +22,7 @@ module AmberLSP::Lookup def perform : String root_path = File.expand_path(@root_path) list_of_index_paths = source_paths(root_path) - key_material = ["project", "docs-entries-v1", "resolved-return-types-v2", @compiler_version, @docs_flags.sort.join("\0")] + key_material = ["project", API_INDEX_CACHE_SCHEMA_VERSION, "docs-entries-v1", "resolved-return-types-v2", @compiler_version, @docs_flags.sort.join("\0")] key_material.concat(@docs_entries) list_of_index_paths.each do |path| @@ -60,6 +62,7 @@ module AmberLSP::Lookup def perform : String key_material = [ "library", + API_INDEX_CACHE_SCHEMA_VERSION, "docs-workspace-multi-entry-v2", "resolved-return-types-v2", @library_name, @@ -77,7 +80,7 @@ module AmberLSP::Lookup end def perform : String - key_material = ["stdlib", "resolved-return-types-v2", @compiler_version, @docs_flags.sort.join("\0")] + key_material = ["stdlib", API_INDEX_CACHE_SCHEMA_VERSION, "resolved-return-types-v2", @compiler_version, @docs_flags.sort.join("\0")] Digest::SHA256.hexdigest(key_material.join("\0")) end end @@ -153,7 +156,8 @@ module AmberLSP::Lookup Dir.mkdir_p(directory_path) layer_path = File.join(directory_path, "#{layer.layer_key}.json") - write_atomically(layer_path, layer.to_json) + cached_layer = layer.layer_kind == "library" ? portable_library_layer(layer) : layer + write_atomically(layer_path, cached_layer.to_json) failure_path = failure_marker_path(layer.layer_kind, layer.layer_name, layer.layer_key) File.delete(failure_path) if File.file?(failure_path) @@ -255,11 +259,120 @@ module AmberLSP::Lookup return nil unless layer.layer_name == expected_name return nil unless layer.layer_key == expected_key - layer + layer.layer_kind == "library" ? resolve_library_layer_paths(layer) : layer rescue ex : JSON::ParseException | IO::Error nil end + private def portable_library_layer(layer : APIIndexLayer) : APIIndexLayer + library_root = File.expand_path(layer.root_path) + list_of_types = layer.list_of_types.map do |type| + APIIndexType.new( + type.name, + type.kind, + type.abstract?, + type.list_of_ancestor_names, + type.list_of_included_module_names, + type.list_of_extended_module_names, + portable_library_source_path(type.location_path, library_root), + type.location_line, + type.source_layer, + type.list_of_instance_methods.map { |method| method_with_source_path(method, portable_library_source_path(method.source_path, library_root)) }, + type.list_of_class_methods.map { |method| method_with_source_path(method, portable_library_source_path(method.source_path, library_root)) }, + type.list_of_macros.map { |method| method_with_source_path(method, portable_library_source_path(method.source_path, library_root)) }, + ) + end + list_of_entry_failures = layer.list_of_entry_failures.map do |failure| + APIIndexEntryFailure.new( + portable_library_source_path(failure.entry_path, library_root), + failure.error, + ) + end + + APIIndexLayer.new( + layer.layer_kind, + layer.layer_name, + layer.layer_key, + ".", + layer.docs_flags, + list_of_types, + list_of_entry_failures, + ) + end + + private def resolve_library_layer_paths(layer : APIIndexLayer) : APIIndexLayer + library_root = File.expand_path(File.join(@project_root_path, "lib", layer.layer_name)) + list_of_types = layer.list_of_types.map do |type| + APIIndexType.new( + type.name, + type.kind, + type.abstract?, + type.list_of_ancestor_names, + type.list_of_included_module_names, + type.list_of_extended_module_names, + resolve_library_source_path(type.location_path, library_root), + type.location_line, + type.source_layer, + type.list_of_instance_methods.map { |method| method_with_source_path(method, resolve_library_source_path(method.source_path, library_root)) }, + type.list_of_class_methods.map { |method| method_with_source_path(method, resolve_library_source_path(method.source_path, library_root)) }, + type.list_of_macros.map { |method| method_with_source_path(method, resolve_library_source_path(method.source_path, library_root)) }, + ) + end + list_of_entry_failures = layer.list_of_entry_failures.map do |failure| + APIIndexEntryFailure.new( + resolve_library_source_path(failure.entry_path, library_root), + failure.error, + ) + end + + APIIndexLayer.new( + layer.layer_kind, + layer.layer_name, + layer.layer_key, + library_root, + layer.docs_flags, + list_of_types, + list_of_entry_failures, + ) + end + + private def method_with_source_path(method : APIIndexMethod, source_path : String) : APIIndexMethod + APIIndexMethod.new( + method.owner, + method.name, + method.method_kind, + method.args_string, + method.declared_return_type, + method.doc_line, + source_path, + method.source_line, + method.source_layer, + method.abstract?, + method.macro?, + method.resolved_return_type, + method.verified_return_type, + method.verification_skip_reason, + ) + end + + private def portable_library_source_path(path : String, library_root : String) : String + return path if path.empty? + return path unless path.starts_with?("/") + + expanded_path = File.expand_path(path) + root_prefix = "#{library_root}/" + return expanded_path[root_prefix.bytesize..] if expanded_path.starts_with?(root_prefix) + return "." if expanded_path == library_root + + raise APIIndexBuildError.new("Library index path is outside #{library_root}: #{path}") + end + + private def resolve_library_source_path(path : String, library_root : String) : String + return path if path.empty? || path.starts_with?("/") + + File.expand_path(path, library_root) + end + private def layer_directory(layer_kind : String, layer_name : String) : String case layer_kind when "stdlib" From 193093f70dce301359e9d5e6441450246803582c Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 02:48:16 -0400 Subject: [PATCH 50/69] fix(lsp): filter foreign docs layer entries --- .../lookup/normalize_crystal_docs_spec.cr | 63 ++++++++++++++ ...crystal_docs_library_source_ownership.json | 83 +++++++++++++++++++ ...crystal_docs_unknown_library_location.json | 17 ++++ .../lookup/build_layered_api_index.cr | 14 +++- src/amber_lsp/lookup/index_cache.cr | 2 +- .../lookup/normalize_crystal_docs.cr | 68 ++++++++++++--- 6 files changed, 233 insertions(+), 14 deletions(-) create mode 100644 spec/fixtures/api_lookup/crystal_docs_library_source_ownership.json create mode 100644 spec/fixtures/api_lookup/crystal_docs_unknown_library_location.json diff --git a/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr b/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr index f5e4d68..1295a4f 100644 --- a/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr +++ b/spec/amber_lsp/lookup/normalize_crystal_docs_spec.cr @@ -1,5 +1,8 @@ require "../spec_helper" +require "../../../src/amber_lsp/lookup/index_cache" +require "../../../src/amber_lsp/lookup/merge_api_index_layers" require "../../../src/amber_lsp/lookup/normalize_crystal_docs" +require "../../../src/amber_lsp/lookup/resolve_api_query" describe "AmberLSP::Lookup::NormalizeCrystalDocs#perform" do it "normalizes nested types, method overloads, module relations, and source locations" do @@ -75,4 +78,64 @@ describe "AmberLSP::Lookup::NormalizeCrystalDocs#perform" do raise Exception.new("Expected the normalized Grant::Collection type") end end + + it "keeps library entries under their owning shard and the stdlib layer" do + s3_layer = normalize_library_source_fixture("awscr-s3") + signer_layer = normalize_library_source_fixture("awscr-signer") + + s3_client = s3_layer.list_of_types.find { |type| type.name == "Awscr::S3::Client" } + signer_client = signer_layer.list_of_types.find { |type| type.name == "Awscr::S3::Client" } + s3_client = s3_client || raise "Expected Awscr::S3::Client in the S3 layer" + signer_client = signer_client || raise "Expected Awscr::S3::Client reopening in the signer layer" + + s3_client.list_of_instance_methods.map(&.name).should eq(["list"]) + signer_client.location_path.should eq("/fixture_app/lib/awscr-signer/src/awscr/s3/client.cr") + signer_client.list_of_instance_methods.map(&.name).should eq(["sign"]) + s3_layer.list_of_types.map(&.name).should contain("Awscr::S3::UnusedS3Type") + signer_layer.list_of_types.map(&.name).should contain("Awscr::Signer::Signer") + s3_layer.list_of_types.map(&.name).should_not contain("HTTP::Headers") + signer_layer.list_of_types.map(&.name).should_not contain("Awscr::S3::UnusedS3Type") + signer_layer.list_of_types.map(&.name).should_not contain("HTTP::Headers") + + merged_layer = AmberLSP::Lookup::MergeAPIIndexLayers.new([s3_layer, signer_layer]).perform + list_resolution = AmberLSP::Lookup::ResolveAPIQuery.new("Awscr::S3::Client#list", [merged_layer]).perform + sign_resolution = AmberLSP::Lookup::ResolveAPIQuery.new("Awscr::S3::Client#sign", [merged_layer]).perform + + list_resolution.list_of_methods.first.source_layer.should eq("awscr-s3") + sign_resolution.list_of_methods.first.source_layer.should eq("awscr-signer") + end + + it "fails when a library docs entry has no known source root" do + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_unknown_library_location.json") + standard_library_root = "/fixture_app/crystal/src" + normalizer = AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + "/fixture_app/lib/awscr-signer", + "library", + "awscr-signer", + "fixture-key", + [] of String, + nil, + {standard_library_root => standard_library_root}, + ) + + error = expect_raises(AmberLSP::Lookup::APIIndexBuildError) { normalizer.perform } + error_message = error.message || raise "Expected a message for the unknown source path" + error_message.should contain("/fixture_app/untracked/private.cr") + end +end + +private def normalize_library_source_fixture(layer_name : String) : AmberLSP::Lookup::APIIndexLayer + fixture_path = File.join(Dir.current, "spec", "fixtures", "api_lookup", "crystal_docs_library_source_ownership.json") + standard_library_root = "/fixture_app/crystal/src" + AmberLSP::Lookup::NormalizeCrystalDocs.new( + File.read(fixture_path), + File.join("/fixture_app/lib", layer_name), + "library", + layer_name, + "#{layer_name}-fixture-key", + [] of String, + nil, + {standard_library_root => standard_library_root}, + ).perform end diff --git a/spec/fixtures/api_lookup/crystal_docs_library_source_ownership.json b/spec/fixtures/api_lookup/crystal_docs_library_source_ownership.json new file mode 100644 index 0000000..f7ec142 --- /dev/null +++ b/spec/fixtures/api_lookup/crystal_docs_library_source_ownership.json @@ -0,0 +1,83 @@ +{ + "repository_name": "library-source-ownership-fixture", + "program": { + "kind": "module", + "full_name": "", + "types": [ + { + "kind": "class", + "name": "Client", + "full_name": "Awscr::S3::Client", + "locations": [ + {"filename": "/fixture_app/lib/awscr-s3/src/awscr/s3/client.cr", "line_number": 1}, + {"filename": "/fixture_app/lib/awscr-signer/src/awscr/s3/client.cr", "line_number": 2} + ], + "instance_methods": [ + { + "name": "list", + "args_string": "() : Array(String)", + "def": {"name": "list", "return_type": "Array(String)"}, + "location": {"filename": "/fixture_app/lib/awscr-s3/src/awscr/s3/client.cr", "line_number": 10} + }, + { + "name": "sign", + "args_string": "() : String", + "def": {"name": "sign", "return_type": "String"}, + "location": {"filename": "/fixture_app/lib/awscr-signer/src/awscr/s3/client.cr", "line_number": 20} + }, + { + "name": "header_value", + "args_string": "() : String | Nil", + "def": {"name": "header_value", "return_type": "String | Nil"}, + "location": {"filename": "/fixture_app/crystal/src/http/headers.cr", "line_number": 30} + } + ] + }, + { + "kind": "class", + "name": "UnusedS3Type", + "full_name": "Awscr::S3::UnusedS3Type", + "locations": [ + {"filename": "/fixture_app/lib/awscr-s3/src/awscr/s3/unused_s3_type.cr", "line_number": 1} + ], + "instance_methods": [ + { + "name": "s3_only", + "args_string": "()", + "location": {"filename": "/fixture_app/lib/awscr-s3/src/awscr/s3/unused_s3_type.cr", "line_number": 2} + } + ] + }, + { + "kind": "class", + "name": "Signer", + "full_name": "Awscr::Signer::Signer", + "locations": [ + {"filename": "/fixture_app/lib/awscr-signer/src/awscr/signer.cr", "line_number": 1} + ], + "instance_methods": [ + { + "name": "sign_request", + "args_string": "()", + "location": {"filename": "/fixture_app/lib/awscr-signer/src/awscr/signer.cr", "line_number": 2} + } + ] + }, + { + "kind": "class", + "name": "Headers", + "full_name": "HTTP::Headers", + "locations": [ + {"filename": "/fixture_app/crystal/src/http/headers.cr", "line_number": 1} + ], + "instance_methods": [ + { + "name": "add", + "args_string": "()", + "location": {"filename": "/fixture_app/crystal/src/http/headers.cr", "line_number": 2} + } + ] + } + ] + } +} diff --git a/spec/fixtures/api_lookup/crystal_docs_unknown_library_location.json b/spec/fixtures/api_lookup/crystal_docs_unknown_library_location.json new file mode 100644 index 0000000..9a9b569 --- /dev/null +++ b/spec/fixtures/api_lookup/crystal_docs_unknown_library_location.json @@ -0,0 +1,17 @@ +{ + "repository_name": "unknown-library-location-fixture", + "program": { + "kind": "module", + "full_name": "", + "types": [ + { + "kind": "class", + "name": "PrivateType", + "full_name": "Untracked::PrivateType", + "locations": [ + {"filename": "/fixture_app/untracked/private.cr", "line_number": 1} + ] + } + ] + } +} diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr index c4ee3cc..f579149 100644 --- a/src/amber_lsp/lookup/build_layered_api_index.cr +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -268,7 +268,11 @@ module AmberLSP::Lookup source_path_mappings = {} of String => String if layer_kind == "library" - workspace = create_library_docs_workspace(File.join(source_root, "src"), list_of_entrypoints) + workspace = create_library_docs_workspace( + File.join(source_root, "src"), + list_of_entrypoints, + crystal_source_root(identity), + ) workspace_root = workspace.root_path docs_working_directory = workspace.root_path list_of_docs_entrypoints = workspace.list_of_entrypoints @@ -411,7 +415,11 @@ module AmberLSP::Lookup ).perform end - private def create_library_docs_workspace(source_root : String, list_of_entrypoints : Array(String)) : CrystalDocsWorkspace + private def create_library_docs_workspace( + source_root : String, + list_of_entrypoints : Array(String), + standard_library_source_root : String, + ) : CrystalDocsWorkspace source_path = File.expand_path(source_root) library_root = File.dirname(source_path) workspace_root = "" @@ -435,6 +443,8 @@ module AmberLSP::Lookup File.join(workspace_root, "src") => source_path, File.join(workspace_root, "lib") => project_lib_path, } + # The self-mapping makes the stdlib source root available to library ownership checks. + source_path_mappings[standard_library_source_root] = standard_library_source_root CrystalDocsWorkspace.new( workspace_root, diff --git a/src/amber_lsp/lookup/index_cache.cr b/src/amber_lsp/lookup/index_cache.cr index 7a22eae..68a4ecc 100644 --- a/src/amber_lsp/lookup/index_cache.cr +++ b/src/amber_lsp/lookup/index_cache.cr @@ -5,7 +5,7 @@ require "random/secure" require "./index_models" module AmberLSP::Lookup - API_INDEX_CACHE_SCHEMA_VERSION = "api-index-cache-v5" + API_INDEX_CACHE_SCHEMA_VERSION = "api-index-cache-v6" class APIIndexBuildError < Exception end diff --git a/src/amber_lsp/lookup/normalize_crystal_docs.cr b/src/amber_lsp/lookup/normalize_crystal_docs.cr index 0bdc465..25b1b98 100644 --- a/src/amber_lsp/lookup/normalize_crystal_docs.cr +++ b/src/amber_lsp/lookup/normalize_crystal_docs.cr @@ -1,4 +1,4 @@ -require "./index_models" +require "./index_cache" require "./parse_api_type_name" module AmberLSP::Lookup @@ -34,15 +34,36 @@ module AmberLSP::Lookup parent.types.each do |docs_type| next if docs_type.locations.empty? - list_of_types << normalize_type(docs_type) + if normalized_type = normalize_type(docs_type) + list_of_types << normalized_type + end append_types_from(docs_type, list_of_types) end end - private def normalize_type(docs_type : CrystalDocsType) : APIIndexType - type_location = docs_type.locations.first? - location_path = type_location ? source_path(type_location.filename) : "" - location_line = type_location ? type_location.line_number : 0 + private def normalize_type(docs_type : CrystalDocsType) : APIIndexType? + list_of_type_locations = docs_type.locations.compact_map do |location| + path = source_path(location.filename) + next unless include_library_source_path?(path) + + {path, location.line_number} + end + list_of_instance_methods = normalize_methods(docs_type, docs_type.instance_methods, "instance", false) + list_of_class_methods = normalize_methods(docs_type, docs_type.class_methods, "class", false) + + normalize_methods(docs_type, docs_type.constructors, "class", false, docs_type.full_name) + list_of_macros = normalize_methods(docs_type, docs_type.macros, "macro", true) + + location_path = "" + location_line = 0 + if type_location = list_of_type_locations.first? + location_path = type_location[0] + location_line = type_location[1] + elsif method_location = (list_of_instance_methods + list_of_class_methods + list_of_macros).first? + location_path = method_location.source_path + location_line = method_location.source_line + else + return nil + end APIIndexType.new( docs_type.full_name, @@ -54,10 +75,9 @@ module AmberLSP::Lookup location_path, location_line, @layer_name, - normalize_methods(docs_type, docs_type.instance_methods, "instance", false), - normalize_methods(docs_type, docs_type.class_methods, "class", false) + - normalize_methods(docs_type, docs_type.constructors, "class", false, docs_type.full_name), - normalize_methods(docs_type, docs_type.macros, "macro", true), + list_of_instance_methods, + list_of_class_methods, + list_of_macros, ) end @@ -71,6 +91,8 @@ module AmberLSP::Lookup docs_methods.compact_map do |docs_method| location = docs_method.location next unless location + path = source_path(location.filename) + next unless include_library_source_path?(path) doc_line = first_doc_line(docs_method.doc) definition = docs_method.definition @@ -88,7 +110,7 @@ module AmberLSP::Lookup method_arguments(docs_method.args_string), declared_return_type, doc_line, - source_path(location.filename), + path, location.line_number, @layer_name, docs_method.abstract?, @@ -97,6 +119,30 @@ module AmberLSP::Lookup end end + private def include_library_source_path?(path : String) : Bool + return true unless @layer_kind == "library" + + expanded_path = File.expand_path(path) + library_root = File.expand_path(@root_path) + return true if path_is_within?(expanded_path, library_root) + + linked_library_root = File.dirname(library_root) + return false if path_is_within?(expanded_path, linked_library_root) + + known_roots = @source_path_mappings.values.map { |root| File.expand_path(root) } + return false if known_roots.any? { |root| path_is_within?(expanded_path, root) } + + raise APIIndexBuildError.new("Library docs entry path is outside known roots: #{path}") + end + + private def path_is_within?(path : String, root : String) : Bool + normalized_root = File.expand_path(root) + return true if path == normalized_root + + root_prefix = normalized_root == "/" ? normalized_root : "#{normalized_root}/" + path.starts_with?(root_prefix) + end + private def included_module_names(docs_type : CrystalDocsType, location_path : String) : Array(String) module_names = docs_type.included_modules.map(&.full_name) type_name = ParseAPITypeName.new(docs_type.full_name).perform From 4fe3ab58cbb8eb2918f0c60c76f1a26a589adc6d Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 02:00:08 -0400 Subject: [PATCH 51/69] build(release): verify Crystal and binary archive checksums --- .github/workflows/release.yml | 147 ++++++++++++++++++++--------- scripts/build_release.sh | 21 ++++- scripts/install_pinned_crystal.sh | 59 ++++++++++++ scripts/install_pinned_minecart.sh | 14 ++- src/amber_lsp/version.cr | 2 +- src/version.cr | 2 +- 6 files changed, 193 insertions(+), 52 deletions(-) create mode 100755 scripts/install_pinned_crystal.sh diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f4b8c10..3eb3aa1 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -23,14 +23,14 @@ jobs: matrix: include: - target: darwin-arm64 - os: macos-latest - crystal_arch: aarch64 + os: macos-26 + runner_arch: arm64 - target: linux-x86_64 - os: ubuntu-latest - crystal_arch: x86_64 + os: ubuntu-24.04 + runner_arch: x86_64 - target: linux-arm64 os: ubuntu-24.04-arm - crystal_arch: aarch64 + runner_arch: aarch64 steps: - name: Checkout code @@ -38,21 +38,18 @@ jobs: with: ref: ${{ github.event.release.tag_name || github.event.inputs.ref }} - - name: Refresh apt package index - # install-crystal runs apt-get install without an update, so a stale - # runner image index 404s on superseded packages. - if: runner.os == 'Linux' - run: sudo apt-get update - - - name: Install Crystal (Linux) - if: startsWith(matrix.target, 'linux-') - uses: crystal-lang/install-crystal@d8ef131ecec0352ce0e39b81b0a6d95def58fe2f # v1 - with: - crystal: latest + - name: Verify runner architecture + run: | + actual_arch="$(uname -m)" + expected_arch="${{ matrix.runner_arch }}" + if [[ "$actual_arch" != "$expected_arch" ]]; then + printf 'Runner architecture mismatch: expected %s, got %s\n' "$expected_arch" "$actual_arch" >&2 + exit 1 + fi + printf 'Verified runner architecture: %s\n' "$actual_arch" - - name: Install Crystal and OpenSSL (macOS) - if: matrix.target == 'darwin-arm64' - run: brew install crystal openssl@3 + - name: Install hash-verified Crystal + run: scripts/install_pinned_crystal.sh "${{ matrix.target }}" - name: Verify release versions agree run: | @@ -66,6 +63,8 @@ jobs: - name: Build pinned Minecart run: scripts/install_pinned_minecart.sh + env: + CRYSTAL: crystal-alpha - name: Install dependencies run: minecart install --production --skip-ai-docs @@ -73,14 +72,14 @@ jobs: - name: Build binaries (Linux) if: startsWith(matrix.target, 'linux-') run: | - crystal build src/amber_cli.cr -o amber --release --static - crystal build src/amber_lsp.cr -o amber-lsp --release --static + crystal-alpha build src/amber_cli.cr -o amber --release --static + crystal-alpha build src/amber_lsp.cr -o amber-lsp --release --static - name: Build binaries (macOS ARM64) if: matrix.target == 'darwin-arm64' run: | - crystal build src/amber_cli.cr -o amber --release - crystal build src/amber_lsp.cr -o amber-lsp --release + crystal-alpha build src/amber_cli.cr -o amber --release + crystal-alpha build src/amber_lsp.cr -o amber-lsp --release - name: Verify binaries run: | @@ -102,7 +101,19 @@ jobs: - name: Create archive run: | mkdir -p dist - tar -czf dist/amber_cli-${{ matrix.target }}.tar.gz amber amber-lsp + if command -v sha256sum >/dev/null 2>&1; then + sha256sum amber amber-lsp > dist/checksums.txt + else + shasum -a 256 amber amber-lsp > dist/checksums.txt + fi + tar -czf dist/amber_cli-${{ matrix.target }}.tar.gz amber amber-lsp -C dist checksums.txt + mkdir -p dist/verify + tar -xzf dist/amber_cli-${{ matrix.target }}.tar.gz -C dist/verify + if command -v sha256sum >/dev/null 2>&1; then + (cd dist/verify && sha256sum --check checksums.txt) + else + (cd dist/verify && shasum -a 256 -c checksums.txt) + fi - name: Calculate checksum id: checksum @@ -123,30 +134,51 @@ jobs: dist/amber_cli-${{ matrix.target }}.tar.gz dist/amber_cli-${{ matrix.target }}.tar.gz.sha256 - upload-assets: - name: Upload Release Assets - needs: build - runs-on: ubuntu-latest - if: github.event_name == 'release' - + package-source: + name: Package checksummed source archive + runs-on: ubuntu-24.04 steps: - - name: Checkout release source + - name: Checkout source uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6 with: - ref: ${{ github.event.release.tag_name }} + ref: ${{ github.event.release.tag_name || github.event.inputs.ref }} + + - name: Verify release version + run: | + version="$(awk '/^version:/ { print $2; exit }' shard.yml)" + requested_ref="${{ github.event.release.tag_name || github.event.inputs.ref }}" + if [[ "$requested_ref" == v* ]]; then + test "v$version" = "$requested_ref" + fi - - name: Package checksummed source archive + - name: Create checksummed source archive run: | version="$(awk '/^version:/ { print $2; exit }' shard.yml)" - test "v$version" = "${{ github.event.release.tag_name }}" mkdir -p dist - git archive --format=tar --prefix="amber_cli-${version}/" HEAD | gzip -n -9 > "dist/amber_cli-source-${version}.tar.gz" + git archive --format=tar.gz --prefix="amber_cli-${version}/" HEAD > "dist/amber_cli-source-${version}.tar.gz" cd dist - if command -v sha256sum >/dev/null 2>&1; then - sha256sum "amber_cli-source-${version}.tar.gz" > "amber_cli-source-${version}.tar.gz.sha256" - else - shasum -a 256 "amber_cli-source-${version}.tar.gz" > "amber_cli-source-${version}.tar.gz.sha256" - fi + sha256sum "amber_cli-source-${version}.tar.gz" > "amber_cli-source-${version}.tar.gz.sha256" + printf 'CI source archive SHA-256: %s\n' "$(cut -d' ' -f1 < "amber_cli-source-${version}.tar.gz.sha256")" + + - name: Upload source archive for preflight and release + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: amber_cli-source + path: | + dist/amber_cli-source-*.tar.gz + dist/amber_cli-source-*.tar.gz.sha256 + + upload-assets: + name: Upload Release Assets + needs: [build, package-source] + runs-on: ubuntu-24.04 + if: github.event_name == 'release' + + steps: + - name: Checkout release source + uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6 + with: + ref: ${{ github.event.release.tag_name }} - name: Download all artifacts uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 @@ -155,8 +187,11 @@ jobs: - name: Verify artifacts exist run: | - set -e + set -euo pipefail + version="$(sed -n 's/^version: *//p' shard.yml | head -1)" for f in \ + "artifacts/amber_cli-source/amber_cli-source-${version}.tar.gz" \ + "artifacts/amber_cli-source/amber_cli-source-${version}.tar.gz.sha256" \ artifacts/amber_cli-darwin-arm64/amber_cli-darwin-arm64.tar.gz \ artifacts/amber_cli-darwin-arm64/amber_cli-darwin-arm64.tar.gz.sha256 \ artifacts/amber_cli-linux-x86_64/amber_cli-linux-x86_64.tar.gz \ @@ -172,6 +207,30 @@ jobs: echo "OK: $f" done + - name: Verify archive hashes and write release notes + run: | + set -euo pipefail + version="$(sed -n 's/^version: *//p' shard.yml | head -1)" + commit_hash="$(git rev-parse HEAD)" + source_archive="amber_cli-source-${version}.tar.gz" + source_checksum_file="artifacts/amber_cli-source/${source_archive}.sha256" + (cd artifacts/amber_cli-source && sha256sum --check "${source_archive}.sha256") + mkdir -p dist + { + printf "# Amber CLI v%s\n\n" "$version" + printf "Release commit: \`%s\`\n\n" "$commit_hash" + printf "Archive SHA-256 values:\n\n" + printf "| Archive | SHA-256 |\n|---|---|\n" + printf "| \`%s\` | \`%s\` |\n" "$source_archive" "$(cut -d' ' -f1 < "$source_checksum_file")" + for target in darwin-arm64 linux-x86_64 linux-arm64; do + archive="amber_cli-${target}.tar.gz" + checksum_file="artifacts/amber_cli-${target}/${archive}.sha256" + (cd "artifacts/amber_cli-${target}" && sha256sum --check "${archive}.sha256") + printf "| \`%s\` | \`%s\` |\n" "$archive" "$(cut -d' ' -f1 < "$checksum_file")" + done + } > dist/release-notes.md + cat dist/release-notes.md + - name: Upload release assets uses: softprops/action-gh-release@c12583777ecdfd3be55c69cf75464299dc01057e # v3 with: @@ -182,9 +241,11 @@ jobs: artifacts/amber_cli-linux-x86_64/amber_cli-linux-x86_64.tar.gz.sha256 artifacts/amber_cli-linux-arm64/amber_cli-linux-arm64.tar.gz artifacts/amber_cli-linux-arm64/amber_cli-linux-arm64.tar.gz.sha256 - dist/amber_cli-source-*.tar.gz - dist/amber_cli-source-*.tar.gz.sha256 + artifacts/amber_cli-source/amber_cli-source-*.tar.gz + artifacts/amber_cli-source/amber_cli-source-*.tar.gz.sha256 tag_name: ${{ github.event.release.tag_name }} fail_on_unmatched_files: true + body_path: dist/release-notes.md + append_body: true env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/scripts/build_release.sh b/scripts/build_release.sh index 6a8391c..8d97907 100755 --- a/scripts/build_release.sh +++ b/scripts/build_release.sh @@ -32,8 +32,8 @@ case "${OS}" in BUILD_LSP="crystal-alpha build src/amber_lsp.cr -o amber-lsp --release" CHECKSUM_CMD="shasum -a 256" if [ "${ARCH}" != "arm64" ]; then - echo "⚠️ Warning: Building for ARM64 on ${ARCH} architecture" - echo " This will create a native build for your current architecture" + echo "Intel macOS is not a release target; build darwin-arm64 on Apple Silicon" >&2 + exit 1 fi ;; "linux") @@ -77,11 +77,23 @@ test -x amber-lsp # Create archive echo "📦 Creating archive..." -tar -czf "${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" amber amber-lsp -git archive --format=tar --prefix="amber_cli-${VERSION}/" HEAD | gzip -n -9 > "${OUTPUT_DIR}/amber_cli-source-${VERSION}.tar.gz" +git archive --format=tar.gz --prefix="amber_cli-${VERSION}/" HEAD > "${OUTPUT_DIR}/amber_cli-source-${VERSION}.tar.gz" # Calculate checksum echo "🔢 Calculating checksum..." +if command -v sha256sum >/dev/null 2>&1; then + sha256sum amber amber-lsp > "${OUTPUT_DIR}/checksums.txt" +else + shasum -a 256 amber amber-lsp > "${OUTPUT_DIR}/checksums.txt" +fi +tar -czf "${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" amber amber-lsp -C "${OUTPUT_DIR}" checksums.txt +mkdir -p "${OUTPUT_DIR}/verify" +tar -xzf "${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" -C "${OUTPUT_DIR}/verify" +if command -v sha256sum >/dev/null 2>&1; then + (cd "${OUTPUT_DIR}/verify" && sha256sum --check checksums.txt) +else + (cd "${OUTPUT_DIR}/verify" && shasum -a 256 -c checksums.txt) +fi cd "${OUTPUT_DIR}" if command -v sha256sum >/dev/null 2>&1; then sha256sum "amber_cli-${TARGET}.tar.gz" > "amber_cli-${TARGET}.tar.gz.sha256" @@ -95,6 +107,7 @@ SHA256=$(cut -d' ' -f1 < "amber_cli-${TARGET}.tar.gz.sha256") echo "" echo "🎉 Build complete!" echo "📁 Output: ${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" +echo "🔒 Commit: $(git -C .. rev-parse HEAD)" echo "🔑 SHA256: ${SHA256}" echo "🔑 Source SHA256: $(cut -d' ' -f1 < "amber_cli-source-${VERSION}.tar.gz.sha256")" echo "" diff --git a/scripts/install_pinned_crystal.sh b/scripts/install_pinned_crystal.sh new file mode 100755 index 0000000..f36225c --- /dev/null +++ b/scripts/install_pinned_crystal.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +set -euo pipefail + +crystal_version="1.21.1" +target="${1:-${TARGET:-}}" +install_root="${CRYSTAL_INSTALL_ROOT:-${RUNNER_TEMP:-${TMPDIR:-/tmp}}/amber-crystal-${crystal_version}-${target}}" + +case "$target" in + darwin-arm64) + archive="crystal-1.21.1-1-darwin-universal.tar.gz" + sha256="ba34fa7fb8bc8b951f12d17e12f69c9d7ccce38900caa4b8a17f1b4f29d37605" + ;; + linux-x86_64) + archive="crystal-1.21.1-1-linux-x86_64.tar.gz" + sha256="08b5779df8cf280c4a799599862bbe4d42103ba082aa73429562713c8a9f3b43" + ;; + linux-arm64) + archive="crystal-1.21.1-1-linux-aarch64.tar.gz" + sha256="a37365b79f58d4cd32b6113bbb9983e21eb41d233667e003c019520e3c245e49" + ;; + *) + printf 'Unsupported Crystal release target: %s\n' "$target" >&2 + exit 2 + ;; +esac + +mkdir -p "$install_root" +archive_path="$install_root/$archive" +download_url="https://github.com/crystal-lang/crystal/releases/download/1.21.1/$archive" +curl --fail --location --silent --show-error --retry 3 --output "$archive_path" "$download_url" + +if command -v sha256sum >/dev/null 2>&1; then + printf '%s %s\n' "$sha256" "$archive_path" | sha256sum --check --status - +else + printf '%s %s\n' "$sha256" "$archive_path" | shasum --algorithm 256 --check --status - +fi + +tar -xzf "$archive_path" -C "$install_root" +compiler_path="$(find "$install_root" -path '*/bin/crystal' -print -quit)" +if [[ -z "$compiler_path" ]]; then + printf 'Crystal %s archive did not contain bin/crystal\n' "$crystal_version" >&2 + exit 1 +fi + +bin_directory="$(dirname "$compiler_path")" +ln -sfn crystal "$bin_directory/crystal-alpha" +version_output="$("$bin_directory/crystal-alpha" --version)" +if [[ "$version_output" != *"Crystal ${crystal_version} "* ]]; then + printf 'Unexpected compiler version in verified archive: %s\n' "$version_output" >&2 + exit 1 +fi + +printf 'Verified Crystal %s archive for %s: %s\n' "$crystal_version" "$target" "$sha256" +printf '%s\n' "$version_output" +if [[ -n "${GITHUB_PATH:-}" ]]; then + printf '%s\n' "$bin_directory" >> "$GITHUB_PATH" +else + printf 'Add this directory to PATH before building: %s\n' "$bin_directory" +fi diff --git a/scripts/install_pinned_minecart.sh b/scripts/install_pinned_minecart.sh index eabcefb..814594c 100755 --- a/scripts/install_pinned_minecart.sh +++ b/scripts/install_pinned_minecart.sh @@ -4,19 +4,27 @@ set -euo pipefail # The source commit is verified by its Git tree before it can run the web smoke. minecart_commit=4fea47ce6561ea0612cf1e0cf6955d782ef27929 minecart_tree=e0a0214c14d71f1031d83b760d426b09509b5d9d +minecart_repository="${MINECART_SOURCE_REPOSITORY:-https://github.com/crimson-knight/shards.git}" +minecart_source_ref="${MINECART_SOURCE_REF:-refs/tags/v2025.11.25.7}" install_root="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/amber-pinned-minecart" mkdir -p "$install_root" git -C "$install_root" init -q -git -C "$install_root" remote add origin https://github.com/crimson-knight/shards.git -git -C "$install_root" fetch --depth=1 origin "$minecart_commit" +if git -C "$install_root" remote get-url origin >/dev/null 2>&1; then + git -C "$install_root" remote set-url origin "$minecart_repository" +else + git -C "$install_root" remote add origin "$minecart_repository" +fi +git -C "$install_root" fetch --depth=1 origin "$minecart_source_ref" git -C "$install_root" checkout -q --detach FETCH_HEAD test "$(git -C "$install_root" rev-parse HEAD)" = "$minecart_commit" test "$(git -C "$install_root" rev-parse 'HEAD^{tree}')" = "$minecart_tree" -make -C "$install_root" bin/minecart CRYSTAL="${CRYSTAL:-crystal}" +make -C "$install_root" bin/minecart bin/shards-alpha CRYSTAL="${CRYSTAL:-crystal-alpha}" test -x "$install_root/bin/minecart" +test -x "$install_root/bin/shards-alpha" "$install_root/bin/minecart" --version | grep -F "Minecart 2025.11.25.7" +"$install_root/bin/shards-alpha" --version | grep -F "Minecart 2025.11.25.7" if [[ -n "${GITHUB_PATH:-}" ]]; then echo "$install_root/bin" >> "$GITHUB_PATH" diff --git a/src/amber_lsp/version.cr b/src/amber_lsp/version.cr index 2545a00..4f1093e 100644 --- a/src/amber_lsp/version.cr +++ b/src/amber_lsp/version.cr @@ -2,7 +2,7 @@ module AmberLSP VERSION = "1.0.0" # The amber_cli release this server ships in, read from shard.yml at compile time. - RELEASE = {{ `shards version "#{__DIR__}/../.."`.chomp.stringify.downcase }} + RELEASE = {{ `minecart version "#{__DIR__}/../.."`.chomp.stringify.downcase }} # The git commit the binary was built from, or "unknown" outside a git checkout. BUILD_COMMIT = {{ `git -C "#{__DIR__}" rev-parse --short=12 HEAD 2>/dev/null || echo unknown`.chomp.stringify }} diff --git a/src/version.cr b/src/version.cr index f00b69e..6661c39 100644 --- a/src/version.cr +++ b/src/version.cr @@ -1,4 +1,4 @@ # :nodoc: module AmberCli - VERSION = {{ `shards version "#{__DIR__}"`.chomp.stringify.downcase }} + VERSION = {{ `minecart version "#{__DIR__}"`.chomp.stringify.downcase }} end From ebf66dd81400e21517aed772285e5a2909f2d959 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 03:13:29 -0400 Subject: [PATCH 52/69] docs(agent): explain project setup and LSP lookup --- .gitignore | 2 ++ README.md | 22 ++++++++++++++++++++-- docs/guides/ai-assistants.md | 30 ++++++++++++++++++++++++++++++ 3 files changed, 52 insertions(+), 2 deletions(-) create mode 100644 docs/guides/ai-assistants.md diff --git a/.gitignore b/.gitignore index 348fcec..20d531f 100644 --- a/.gitignore +++ b/.gitignore @@ -22,6 +22,8 @@ Thumbs.db /docs/** !/docs/*.md !/docs/.gitkeep +!/docs/guides/ +!/docs/guides/*.md # Build artifacts /amber diff --git a/README.md b/README.md index f705898..60b00ad 100644 --- a/README.md +++ b/README.md @@ -37,13 +37,13 @@ are selected by the installed core catalog, not pinned by this tap. ### Direct release archive -The published CLI `2.0.6` has `darwin-arm64`, `linux-x86_64`, and +The CLI `2.0.7` release has `darwin-arm64`, `linux-x86_64`, and `linux-arm64` binary archives. A direct archive does not install Minecart or the native dependencies, so Homebrew is the beginner path. Windows x86-64 has no release archive. ```bash -version=v2.0.6 +version=v2.0.7 platform=darwin-arm64 asset="amber_cli-${platform}.tar.gz" @@ -200,6 +200,14 @@ See the [LSP setup guide](https://github.com/amberframework/amber/blob/v2.0.0-be ## AI agent setup +Run these commands in an Amber project to install or refresh the agent hooks +and check their setup: + +```bash +amber setup:agent +amber doctor +``` + `amber new` runs `amber setup:agent` automatically. Run it again after upgrading Amber CLI to refresh `.amber/agent_setup.json`, `.amber/amber-agent-hook`, `.claude/settings.json`, `.codex/hooks.json`, and the @@ -218,6 +226,16 @@ worktrees, and API lookup index freshness. Doctor reads the agent trust records without changing them. Claude and Codex may need the project and its hooks trusted before their project-level hooks can run. +Use `amber-lsp` to look up an API or get a hint for compiler output: + +```bash +amber-lsp lookup 'Dir.mkdir_p' +crystal-alpha build src/my_app.cr 2>&1 | amber-lsp hint +``` + +See the [AI assistant setup guide](docs/guides/ai-assistants.md) for the short +project setup steps. + The post-edit hook formats and checks only the changed Crystal file with `amber-lsp --check`; the stop hook asks `crystal-alpha watch build` for compiler feedback, or checks the target in diff --git a/docs/guides/ai-assistants.md b/docs/guides/ai-assistants.md new file mode 100644 index 0000000..a4c79f5 --- /dev/null +++ b/docs/guides/ai-assistants.md @@ -0,0 +1,30 @@ +# AI assistant setup + +Amber CLI adds project hooks for Claude Code and Codex. From the project root, +run: + +```bash +amber setup:agent +amber doctor +``` + +`setup:agent` creates or refreshes the hook files and merges them with the +project's existing Claude and Codex settings. Review and commit the generated +files. `amber doctor` reports whether the hooks, trust settings, LSP binary, +and API lookup index are ready. + +Install a checksum-verified `amber-lsp` binary and `crystal-alpha` to enable +Crystal edit checks. The pre-tool hook checks the edited Crystal file before +the agent writes it, and the post-edit hook formats and analyzes that file. +If setup is incomplete, run `amber setup:agent` again and follow the readiness +message. + +Use API lookup and compiler hints from the project root: + +```bash +amber-lsp lookup 'Dir.mkdir_p' +crystal-alpha build src/my_app.cr 2>&1 | amber-lsp hint +``` + +`lookup` searches the Amber and Crystal API index. `hint` reads compiler output +from standard input and prints a matching API card hint when one is available. From bc08f832b7777fdda8339c41b672e55537961259 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 04:11:44 -0400 Subject: [PATCH 53/69] feat(lsp): load bundled API cards --- spec/amber_lsp/lookup/api_cards_spec.cr | 116 ++++++++++++++ src/amber_lsp/cards/embedded_api_cards.cr | 6 + src/amber_lsp/lookup/api_cards.cr | 186 +++++++++++++++++++--- 3 files changed, 287 insertions(+), 21 deletions(-) create mode 100644 src/amber_lsp/cards/embedded_api_cards.cr diff --git a/spec/amber_lsp/lookup/api_cards_spec.cr b/spec/amber_lsp/lookup/api_cards_spec.cr index ad083cb..e1f6bdb 100644 --- a/spec/amber_lsp/lookup/api_cards_spec.cr +++ b/spec/amber_lsp/lookup/api_cards_spec.cr @@ -91,6 +91,87 @@ describe AmberLSP::Lookup::LoadAPICards do matches.map(&.card_version).should eq(["1.5.0", "1.5.0"]) end end + + it "loads a version-compatible installed bundled card from the real executable prefix" do + with_bundled_card_project do |project, _executable_path, prefix| + real_prefix = File.join(prefix, "real") + real_bin_path = File.join(real_prefix, "bin") + Dir.mkdir_p(real_bin_path) + real_executable_path = File.join(real_bin_path, "amber-lsp") + File.write(real_executable_path, "binary") + symlink_path = File.join(prefix, "bin", "amber-lsp") + File.delete(symlink_path) + File.symlink(real_executable_path, symlink_path) + installed_cards_path = File.join(real_prefix, "share", "amber_cli", "api") + Dir.mkdir_p(installed_cards_path) + card_path = File.join(installed_cards_path, "crystal.yml") + File.write(card_path, bundled_test_card("installed hint", ">= 1.21.0, < 2.0.0")) + + cards = AmberLSP::Lookup::LoadAPICards.new( + project, + symlink_path, + [{"crystal.yml", bundled_test_card("compiled fallback hint", ">= 1.21.0, < 2.0.0")}], + ).perform + loaded_card = cards.list_of_cards.find { |card| card.card.library == "crystal" } + + cards.list_of_errors.should be_empty + loaded_card.should_not be_nil + if loaded = loaded_card + loaded.source_path.should eq(File.realpath(card_path)) + loaded.resolved_version.should eq("1.21.0") + loaded.origin.should eq(AmberLSP::Lookup::APICardOrigin::Bundled) + else + fail("expected the installed Crystal API card") + end + cards.matching_error_hint_matches("card conflict").map(&.error_hint.hint).should eq(["installed hint"]) + end + end + + it "uses the compiled card when an installed card does not apply to this Crystal version" do + with_bundled_card_project do |project, executable_path, prefix| + installed_cards_path = File.join(prefix, "share", "amber_cli", "api") + Dir.mkdir_p(installed_cards_path) + File.write( + File.join(installed_cards_path, "crystal.yml"), + bundled_test_card("future installed hint", ">= 99.0.0"), + ) + embedded_cards = [{"crystal.yml", bundled_test_card("compiled hint", ">= 1.21.0, < 2.0.0")}] + + cards = AmberLSP::Lookup::LoadAPICards.new(project, executable_path, embedded_cards).perform + loaded_card = cards.list_of_cards.find { |card| card.card.library == "crystal" } + + cards.list_of_errors.should be_empty + loaded_card.should_not be_nil + if loaded = loaded_card + loaded.source_path.should eq("embedded:crystal.yml") + else + fail("expected the compiled Crystal API card") + end + cards.matching_error_hint_matches("card conflict").map(&.error_hint.hint).should eq(["compiled hint"]) + end + end + + it "gives project and library hints precedence over conflicting bundled hints" do + with_bundled_card_project do |project, executable_path, _prefix| + write_test_card(project, "fixture_project", "project pattern", "project hint") + library_root = File.join(project, "lib", "example") + Dir.mkdir_p(File.join(library_root, ".amber-lsp", "api")) + File.write( + File.join(library_root, ".amber-lsp", "api", "example.yml"), + card_yaml("example", "library pattern", "library hint", ">= 0.3.0"), + ) + File.write( + File.join(project, "shard.lock"), + "shards:\n example:\n version: 0.3.5\n", + ) + embedded_cards = [{"crystal.yml", bundled_test_card("bundled hint", ">= 1.21.0")}] + + cards = AmberLSP::Lookup::LoadAPICards.new(project, executable_path, embedded_cards).perform + + cards.matching_error_hint_matches("project pattern").map(&.error_hint.hint).should eq(["project hint"]) + cards.matching_error_hint_matches("library pattern").map(&.error_hint.hint).should eq(["library hint"]) + end + end end private def with_api_card_project(&) @@ -114,3 +195,38 @@ private def with_capture_card_project(&) yield project end end + +private def with_bundled_card_project(&) + with_tempdir do |directory| + project = File.join(directory, "project") + prefix = File.join(directory, "prefix") + Dir.mkdir_p(project) + Dir.mkdir_p(File.join(prefix, "bin")) + File.write(File.join(project, "shard.yml"), "name: fixture_project\nversion: 1.5.0\n") + executable_path = File.join(prefix, "bin", "amber-lsp") + File.write(executable_path, "binary") + yield project, executable_path, prefix + end +end + +private def bundled_test_card(hint : String, requirement : String) : String + card_yaml("crystal", "card conflict", hint, requirement) +end + +private def card_yaml(library : String, pattern : String, hint : String, requirement : String = ">= 1.0.0") : String + <<-YAML + card_version: 1 + library: #{library} + applies_to: "#{requirement}" + error_hints: + - pattern: "#{pattern}" + hint: "#{hint}" + example: "#{hint}" + YAML +end + +private def write_test_card(project : String, library : String, pattern : String, hint : String) : Nil + api_path = File.join(project, ".amber-lsp", "api") + Dir.mkdir_p(api_path) + File.write(File.join(api_path, "#{library}.yml"), card_yaml(library, pattern, hint)) +end diff --git a/src/amber_lsp/cards/embedded_api_cards.cr b/src/amber_lsp/cards/embedded_api_cards.cr new file mode 100644 index 0000000..3003188 --- /dev/null +++ b/src/amber_lsp/cards/embedded_api_cards.cr @@ -0,0 +1,6 @@ +module AmberLSP::Lookup + alias EmbeddedAPICardYAML = Tuple(String, String) + + # :nodoc: + EMBEDDED_API_CARD_YAMLS = [] of EmbeddedAPICardYAML +end diff --git a/src/amber_lsp/lookup/api_cards.cr b/src/amber_lsp/lookup/api_cards.cr index a0f9550..02397c7 100644 --- a/src/amber_lsp/lookup/api_cards.cr +++ b/src/amber_lsp/lookup/api_cards.cr @@ -3,8 +3,15 @@ require "json" require "yaml" require "./source_models" +require "../cards/embedded_api_cards" module AmberLSP::Lookup + enum APICardOrigin + Project + Library + Bundled + end + struct APICardNote include YAML::Serializable include JSON::Serializable @@ -41,8 +48,14 @@ module AmberLSP::Lookup getter card : APICard getter source_path : String getter resolved_version : String - - def initialize(@card : APICard, @source_path : String, @resolved_version : String) + getter origin : APICardOrigin + + def initialize( + @card : APICard, + @source_path : String, + @resolved_version : String, + @origin : APICardOrigin, + ) end end @@ -51,8 +64,14 @@ module AmberLSP::Lookup getter card_library : String getter card_version : String getter error_hint : APICardErrorHint - - def initialize(@card_library : String, @card_version : String, @error_hint : APICardErrorHint) + getter origin : APICardOrigin + + def initialize( + @card_library : String, + @card_version : String, + @error_hint : APICardErrorHint, + @origin : APICardOrigin, + ) end end @@ -83,7 +102,12 @@ module AmberLSP::Lookup def matching_notes(query : String) : Array(APICardNote) query_method_name = query.split(/[.#]/).last? - @list_of_cards.flat_map do |loaded_card| + matching_cards = @list_of_cards.select do |loaded_card| + loaded_card.card.notes.select do |note| + note.symbol == query || note.symbol == query_method_name + end.any? + end + prefer_non_bundled(matching_cards).flat_map do |loaded_card| loaded_card.card.notes.select do |note| note.symbol == query || note.symbol == query_method_name end @@ -95,7 +119,12 @@ module AmberLSP::Lookup end def matching_error_hint_matches(error_text : String) : Array(APICardErrorHintMatch) - @list_of_cards.flat_map do |loaded_card| + matching_cards = @list_of_cards.select do |loaded_card| + loaded_card.card.error_hints.any? do |hint| + Regex.new(hint.pattern).match(error_text) != nil + end + end + prefer_non_bundled(matching_cards).flat_map do |loaded_card| loaded_card.card.error_hints.compact_map do |hint| match = Regex.new(hint.pattern).match(error_text) next unless match @@ -105,11 +134,21 @@ module AmberLSP::Lookup substitute_captures(hint.hint, match), substitute_captures(hint.example, match), ) - APICardErrorHintMatch.new(loaded_card.card.library, loaded_card.resolved_version, interpolated_hint) + APICardErrorHintMatch.new( + loaded_card.card.library, + loaded_card.resolved_version, + interpolated_hint, + loaded_card.origin, + ) end end end + private def prefer_non_bundled(cards : Array(LoadedAPICard)) : Array(LoadedAPICard) + non_bundled_cards = cards.reject { |loaded_card| loaded_card.origin == APICardOrigin::Bundled } + non_bundled_cards.empty? ? cards : non_bundled_cards + end + private def substitute_captures(template : String, match : Regex::MatchData) : String template.gsub(/\{([^{}]+)\}/) do |placeholder| capture_name = placeholder.byte_slice(1, placeholder.bytesize - 2) @@ -200,13 +239,22 @@ module AmberLSP::Lookup end class LoadAPICards - def initialize(@project_root_path : String) + def initialize( + @project_root_path : String, + @executable_path : String? = Process.executable_path, + @embedded_card_yamls : Array(EmbeddedAPICardYAML) = EMBEDDED_API_CARD_YAMLS, + ) @project_root_path = File.expand_path(@project_root_path) end def perform : APICardCollection project_manifest = read_project_manifest - list_of_targets = [CardSource.new(@project_root_path, project_manifest.name, project_manifest.version)] + list_of_targets = [CardSource.new( + @project_root_path, + project_manifest.name, + project_manifest.version, + APICardOrigin::Project, + )] list_of_targets.concat(library_card_sources) list_of_cards = [] of LoadedAPICard @@ -215,18 +263,12 @@ module AmberLSP::Lookup next if target.library_name.empty? card_paths(target.root_path).each do |path| - begin - card = APICard.from_yaml(File.read(path)) - validate_card(card, target, path) - next unless APIVersionRequirement.new(card.applies_to).matches?(target.resolved_version) - - list_of_cards << LoadedAPICard.new(card, path, target.resolved_version) - rescue ex : YAML::ParseException | IO::Error | ArgumentError - list_of_errors << "#{path}: #{ex.message}" - end + load_card_file(target, path, list_of_cards, list_of_errors) end end + load_bundled_cards(list_of_cards, list_of_errors) + duplicate_library_cards(list_of_cards).each do |library_name| list_of_errors << "More than one API card applies to #{library_name}" list_of_cards.reject! { |loaded_card| loaded_card.card.library == library_name } @@ -253,7 +295,7 @@ module AmberLSP::Lookup library_root = File.join(@project_root_path, "lib", library_name) next unless File.directory?(library_root) - CardSource.new(library_root, library_name, locked_shard.version) + CardSource.new(library_root, library_name, locked_shard.version, APICardOrigin::Library) end end @@ -261,6 +303,102 @@ module AmberLSP::Lookup Dir.glob(File.join(root_path, ".amber-lsp", "api", "*.yml")).sort end + private def load_card_file( + target : CardSource, + path : String, + list_of_cards : Array(LoadedAPICard), + list_of_errors : Array(String), + ) : Nil + card = APICard.from_yaml(File.read(path)) + validate_card(card, target, path) + return unless APIVersionRequirement.new(card.applies_to).matches?(target.resolved_version) + + list_of_cards << LoadedAPICard.new(card, path, target.resolved_version, target.origin) + rescue ex : YAML::ParseException | IO::Error | ArgumentError + list_of_errors << "#{path}: #{ex.message}" + end + + private def load_bundled_cards( + list_of_cards : Array(LoadedAPICard), + list_of_errors : Array(String), + ) : Nil + installed_paths = installed_bundled_card_paths + return if installed_paths.empty? && @embedded_card_yamls.empty? + + crystal_version = read_crystal_version + libraries_with_precedence = list_of_cards.map(&.card.library) + installed_paths.each do |path| + load_bundled_card_file( + File.read(path), + path, + crystal_version, + list_of_cards, + list_of_errors, + libraries_with_precedence, + ) + rescue ex : IO::Error + list_of_errors << "#{path}: #{ex.message}" + end + + @embedded_card_yamls.each do |source_path, yaml| + load_bundled_card_file( + yaml, + "embedded:#{source_path}", + crystal_version, + list_of_cards, + list_of_errors, + list_of_cards.map(&.card.library), + ) + end + rescue ex : IO::Error | ArgumentError + list_of_errors << "Bundled API cards: #{ex.message || ex.class.to_s}" + end + + private def load_bundled_card_file( + yaml : String, + path : String, + crystal_version : String, + list_of_cards : Array(LoadedAPICard), + list_of_errors : Array(String), + libraries_with_precedence : Array(String), + ) : Nil + card = APICard.from_yaml(yaml) + target = CardSource.new("", card.library, crystal_version, APICardOrigin::Bundled) + validate_card(card, target, path) + return if libraries_with_precedence.includes?(card.library) + return unless APIVersionRequirement.new(card.applies_to).matches?(crystal_version) + + list_of_cards << LoadedAPICard.new(card, path, crystal_version, APICardOrigin::Bundled) + rescue ex : YAML::ParseException | ArgumentError + list_of_errors << "#{path}: #{ex.message}" + end + + private def installed_bundled_card_paths : Array(String) + return [] of String unless executable_path = @executable_path + + real_executable_path = File.realpath(executable_path) + prefix_path = File.dirname(File.dirname(real_executable_path)) + Dir.glob(File.join(prefix_path, "share", "amber_cli", "api", "*.yml")).sort + rescue ex : IO::Error + [] of String + end + + private def read_crystal_version : String + output = IO::Memory.new + errors = IO::Memory.new + status = Process.run("crystal-alpha", ["--version"], output: output, error: errors) + unless status.success? + raise ArgumentError.new("crystal-alpha --version failed: #{errors.to_s.strip}") + end + + version_match = output.to_s.match(/\bCrystal\s+(\d+\.\d+(?:\.\d+)?(?:-[0-9A-Za-z.-]+)?)/) + if version_match + version_match[1] + else + raise ArgumentError.new("could not parse Crystal version from #{output.to_s.strip.inspect}") + end + end + private def validate_card(card : APICard, target : CardSource, path : String) : Nil raise ArgumentError.new("card_version must be 1") unless card.card_version == 1 raise ArgumentError.new("library must be #{target.library_name}") unless card.library == target.library_name @@ -305,8 +443,14 @@ module AmberLSP::Lookup getter root_path : String getter library_name : String getter resolved_version : String - - def initialize(@root_path : String, @library_name : String, @resolved_version : String) + getter origin : APICardOrigin + + def initialize( + @root_path : String, + @library_name : String, + @resolved_version : String, + @origin : APICardOrigin, + ) end end end From a28008ba81410ef0a83ff44ce30edca310542138 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 04:20:58 -0400 Subject: [PATCH 54/69] fix(build): fall back to shards when Minecart is not installed The version macros required minecart on PATH, so crystal-alpha spec and every build failed with exit 127 for contributors and CI jobs without Minecart. Prefer minecart, then shards-alpha, then shards; all three read the same shard.yml version. --- src/amber_lsp/version.cr | 3 ++- src/version.cr | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/src/amber_lsp/version.cr b/src/amber_lsp/version.cr index 4f1093e..5577aa5 100644 --- a/src/amber_lsp/version.cr +++ b/src/amber_lsp/version.cr @@ -2,7 +2,8 @@ module AmberLSP VERSION = "1.0.0" # The amber_cli release this server ships in, read from shard.yml at compile time. - RELEASE = {{ `minecart version "#{__DIR__}/../.."`.chomp.stringify.downcase }} + # Minecart is preferred; shards-alpha or shards answers when Minecart is not installed. + RELEASE = {{ `minecart version "#{__DIR__}/../.." 2>/dev/null || shards-alpha version "#{__DIR__}/../.." 2>/dev/null || shards version "#{__DIR__}/../.."`.chomp.stringify.downcase }} # The git commit the binary was built from, or "unknown" outside a git checkout. BUILD_COMMIT = {{ `git -C "#{__DIR__}" rev-parse --short=12 HEAD 2>/dev/null || echo unknown`.chomp.stringify }} diff --git a/src/version.cr b/src/version.cr index 6661c39..152b23b 100644 --- a/src/version.cr +++ b/src/version.cr @@ -1,4 +1,5 @@ # :nodoc: module AmberCli - VERSION = {{ `minecart version "#{__DIR__}"`.chomp.stringify.downcase }} + # Minecart is preferred; shards-alpha or shards answers when Minecart is not installed. + VERSION = {{ `minecart version "#{__DIR__}" 2>/dev/null || shards-alpha version "#{__DIR__}" 2>/dev/null || shards version "#{__DIR__}"`.chomp.stringify.downcase }} end From a638d82407fc2f55ff36c1b9bdd4d194cf764ec2 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 04:51:04 -0400 Subject: [PATCH 55/69] fix(agent): require API lookup guidance --- README.md | 2 + docs/guides/ai-assistants.md | 2 + spec/commands/agent_hook_spec.cr | 10 +++-- spec/commands/doctor_spec.cr | 41 ++++++++++++++++++- spec/commands/setup_agent_spec.cr | 8 ++-- src/amber_cli/agent/agent_setup_guidance.cr | 7 ++++ .../check_amber_agent_setup_command.cr | 40 ++++++++++++++++++ src/amber_cli/commands/setup_agent_command.cr | 22 ++++++---- .../templates/agent/amber-agent-hook | 6 ++- 9 files changed, 121 insertions(+), 17 deletions(-) create mode 100644 src/amber_cli/agent/agent_setup_guidance.cr diff --git a/README.md b/README.md index 60b00ad..a56a591 100644 --- a/README.md +++ b/README.md @@ -226,6 +226,8 @@ worktrees, and API lookup index freshness. Doctor reads the agent trust records without changing them. Claude and Codex may need the project and its hooks trusted before their project-level hooks can run. +Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints. + Use `amber-lsp` to look up an API or get a hint for compiler output: ```bash diff --git a/docs/guides/ai-assistants.md b/docs/guides/ai-assistants.md index a4c79f5..16823c4 100644 --- a/docs/guides/ai-assistants.md +++ b/docs/guides/ai-assistants.md @@ -19,6 +19,8 @@ the agent writes it, and the post-edit hook formats and analyzes that file. If setup is incomplete, run `amber setup:agent` again and follow the readiness message. +Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints. + Use API lookup and compiler hints from the project root: ```bash diff --git a/spec/commands/agent_hook_spec.cr b/spec/commands/agent_hook_spec.cr index b0da565..5e6fe2b 100644 --- a/spec/commands/agent_hook_spec.cr +++ b/spec/commands/agent_hook_spec.cr @@ -16,7 +16,7 @@ module AgentHookSpecHelper File.write(".amber/agent_setup.json", AmberCLI::Agent::AgentSetupManifest.new( AmberCli::VERSION, "1.0.0", - "4", + AmberCLI::Agent::AgentSetupGuidance::GENERATED_HOOK_VERSION, ).to_pretty_json + "\n") File.write(".amber/claude-marketplace/.claude-plugin/marketplace.json", <<-JSON) { @@ -150,7 +150,7 @@ describe "amber-agent-hook readiness and preflight" do session[0].should eq(0) session[1].should contain("Amber agent setup is OK.") session[1].should contain("amber-lsp version: 1.0.0") - session[1].should contain("Before using a library API you are not sure of, run `amber-lsp lookup 'Type.method'` (or the LSP tool's workspaceSymbol/hover).") + session[1].should contain("Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints.") end end @@ -339,7 +339,11 @@ describe "amber-agent-hook readiness and preflight" do first_check = AgentHookSpecHelper.run_hook(fixture, "check") first_check[0].should eq(0) first_calls = File.read(fixture.lsp_log).lines.size - setup_manifest = AmberCLI::Agent::AgentSetupManifest.new(AmberCli::VERSION, "1.0.0", "4") + setup_manifest = AmberCLI::Agent::AgentSetupManifest.new( + AmberCli::VERSION, + "1.0.0", + AmberCLI::Agent::AgentSetupGuidance::GENERATED_HOOK_VERSION, + ) File.write(".amber/agent_setup.json", setup_manifest.to_json + "\n ") second_check = AgentHookSpecHelper.run_hook(fixture, "check") diff --git a/spec/commands/doctor_spec.cr b/spec/commands/doctor_spec.cr index 9a6e79b..a27fbbc 100644 --- a/spec/commands/doctor_spec.cr +++ b/spec/commands/doctor_spec.cr @@ -1,15 +1,33 @@ require "../amber_cli_spec" require "../../src/amber_cli/commands/check_amber_agent_setup_command" require "../../src/amber_cli/agent/install_claude_amber_lsp_plugin" +require "../../src/amber_cli/agent/agent_setup_manifest" +require "../../src/amber_cli/agent/agent_setup_guidance" module DoctorCommandSpecHelper def self.create_agent_settings(project_root : String) : String Dir.mkdir_p(File.join(project_root, ".claude")) Dir.mkdir_p(File.join(project_root, ".codex")) + Dir.mkdir_p(File.join(project_root, ".amber")) claude_settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", true).perform codex_settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", false).perform File.write(File.join(project_root, ".claude/settings.json"), claude_settings) File.write(File.join(project_root, ".codex/hooks.json"), codex_settings) + setup_manifest = AmberCLI::Agent::AgentSetupManifest.new( + "2.0.7", + "1.0.0", + AmberCLI::Agent::AgentSetupGuidance::GENERATED_HOOK_VERSION, + ) + File.write(File.join(project_root, ".amber/agent_setup.json"), setup_manifest.to_pretty_json + "\n") + File.write(File.join(project_root, ".amber/amber-agent-hook"), AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION + "\n") + agent_loop_instructions = [ + AmberCLI::Agent::AgentSetupGuidance::DOCUMENT_START, + "## Agent loop", + AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION, + AmberCLI::Agent::AgentSetupGuidance::DOCUMENT_END, + ].join("\n") + "\n" + File.write(File.join(project_root, "CLAUDE.md"), agent_loop_instructions) + File.write(File.join(project_root, "AGENTS.md"), agent_loop_instructions) codex_settings end @@ -81,6 +99,25 @@ describe "amber doctor" do end end + it "reports old generated lookup instructions as outdated" do + SpecHelper.within_temp_directory do |project_root| + DoctorCommandSpecHelper.create_agent_settings(project_root) + Dir.mkdir_p(File.join(project_root, ".amber")) + old_guidance = "Before using a library API you are not sure of, run `amber-lsp lookup 'Type.method'`." + File.write(File.join(project_root, ".amber/agent_setup.json"), <<-JSON) + {"amber_cli_version":"2.0.7","minimum_amber_lsp_version":"1.0.0","generated_hook_version":"4"} + JSON + File.write(File.join(project_root, ".amber/amber-agent-hook"), "#!/bin/sh\n#{old_guidance}\n") + File.write(File.join(project_root, "CLAUDE.md"), "\n#{old_guidance}\n\n") + File.write(File.join(project_root, "AGENTS.md"), "\n#{old_guidance}\n\n") + + report = AmberCLI::Commands::CheckAmberAgentSetupCommand.new("doctor", project_root, nil, false).perform + + report.exit_code.should eq(1) + report.to_s.should contain("[FAIL] Generated agent instructions are out of date; run `amber setup:agent`.") + end + end + it "reports a ready project with exit status 0 when hooks, trust, plugins, and the API index pass" do SpecHelper.within_temp_directory do |project_root| home_directory = File.join(project_root, "home") @@ -100,7 +137,7 @@ describe "amber doctor" do DoctorCommandSpecHelper.write_executable(File.join(tools_directory, "codex"), "#!/bin/sh\nexit 0\n") DoctorCommandSpecHelper.write_executable( File.join(project_root, ".amber/amber-agent-hook"), - "#!/bin/sh\nprintf '%s\\n' 'Amber agent setup is ready.'\nexit 0\n", + "#!/bin/sh\n# #{AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION}\nprintf '%s\\n' 'Amber agent setup is ready.'\nexit 0\n", ) lsp_path = File.join(tools_directory, "amber-lsp") DoctorCommandSpecHelper.write_executable( @@ -166,7 +203,7 @@ describe "amber doctor" do DoctorCommandSpecHelper.write_executable(File.join(tools_directory, "codex"), "#!/bin/sh\nexit 0\n") DoctorCommandSpecHelper.write_executable( File.join(project_root, ".amber/amber-agent-hook"), - "#!/bin/sh\nprintf '%s\\n' 'Amber agent setup is ready.'\nexit 0\n", + "#!/bin/sh\n# #{AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION}\nprintf '%s\\n' 'Amber agent setup is ready.'\nexit 0\n", ) lsp_path = File.join(tools_directory, "amber-lsp") DoctorCommandSpecHelper.write_executable( diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr index f22aa56..866343b 100644 --- a/spec/commands/setup_agent_spec.cr +++ b/spec/commands/setup_agent_spec.cr @@ -124,13 +124,13 @@ describe "amber setup:agent" do File.read("AGENTS.md").scan(/amber-agent-loop:start/).size.should eq(1) first_instructions.should contain("# Existing Claude instructions") first_instructions.should contain("crystal-alpha spec --affected") - first_instructions.should contain("amber-lsp lookup 'Type.method'") + first_instructions.should contain("Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints.") first_instructions.should contain("run `amber setup:agent` before editing") first_script.should contain("build --no-codegen 'src/custom_entry.cr'") setup_manifest = AmberCLI::Agent::AgentSetupManifest.from_json(File.read(".amber/agent_setup.json")) setup_manifest.amber_cli_version.should eq(AmberCli::VERSION) setup_manifest.minimum_amber_lsp_version.should eq("1.0.0") - setup_manifest.generated_hook_version.should eq("4") + setup_manifest.generated_hook_version.should eq("5") File.file?(".amber/claude-marketplace/amber-lsp/.lsp.json").should be_true File.file?(".lsp.json").should be_false File.info(".amber/amber-agent-hook").permissions.to_i.&(0o111).should_not eq(0) @@ -308,7 +308,7 @@ describe "amber setup:agent" do Dir.mkdir_p("src") File.write("shard.yml", "name: my_app\n") File.write("src/my_app.cr", "puts :ok\n") - older_instructions = "# Keep this\n\n\nUse `crystal spec --affected` when available.\n\n" + older_instructions = "# Keep this\n\n\n## Agent loop\n\nUse `crystal spec --affected` when available.\nBefore using a library API you are not sure of, run\n`amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover.\nThe SessionStart hook reports whether setup is complete.\n\n" File.write("CLAUDE.md", older_instructions) File.write("AGENTS.md", older_instructions) @@ -318,6 +318,8 @@ describe "amber setup:agent" do first_instructions.should contain("# Keep this") first_instructions.should contain("Use `crystal-alpha spec --affected`") first_instructions.should_not contain("Use `crystal spec --affected`") + first_instructions.should contain("Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints.") + first_instructions.should_not contain("Before using a library API you are not sure of") File.read("AGENTS.md").should eq(first_instructions) command.execute diff --git a/src/amber_cli/agent/agent_setup_guidance.cr b/src/amber_cli/agent/agent_setup_guidance.cr new file mode 100644 index 0000000..bb3c63b --- /dev/null +++ b/src/amber_cli/agent/agent_setup_guidance.cr @@ -0,0 +1,7 @@ +module AmberCLI::Agent::AgentSetupGuidance + GENERATED_HOOK_VERSION = "5" + DOCUMENT_START = "" + DOCUMENT_END = "" + + REQUIRED_LOOKUP_INSTRUCTION = "Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints." +end diff --git a/src/amber_cli/commands/check_amber_agent_setup_command.cr b/src/amber_cli/commands/check_amber_agent_setup_command.cr index 2e957fd..f1922e3 100644 --- a/src/amber_cli/commands/check_amber_agent_setup_command.cr +++ b/src/amber_cli/commands/check_amber_agent_setup_command.cr @@ -1,6 +1,8 @@ require "json" require "../core/base_command" require "../agent/hook_settings" +require "../agent/agent_setup_manifest" +require "../agent/agent_setup_guidance" require "../agent/find_agent_hook_ignore_rules" module AmberCLI::Commands @@ -109,6 +111,7 @@ module AmberCLI::Commands list_of_checks = [] of DoctorCheck list_of_checks << check_project_hooks(".claude/settings.json", true) list_of_checks << check_project_hooks(".codex/hooks.json", false) + list_of_checks << check_generated_agent_instructions list_of_checks.concat(build_agent_readiness_checks) list_of_checks.concat(build_claude_trust_checks) list_of_checks.concat(build_claude_plugin_checks) @@ -174,6 +177,43 @@ module AmberCLI::Commands end end + private def check_generated_agent_instructions : DoctorCheck + setup_manifest_path = File.join(@project_root, ".amber/agent_setup.json") + hook_path = File.join(@project_root, ".amber/amber-agent-hook") + required_instruction = AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION + has_current_manifest = false + if File.file?(setup_manifest_path) + setup_manifest = AmberCLI::Agent::AgentSetupManifest.from_json(File.read(setup_manifest_path)) + has_current_manifest = setup_manifest.generated_hook_version == AmberCLI::Agent::AgentSetupGuidance::GENERATED_HOOK_VERSION + end + + has_current_hook = File.file?(hook_path) && File.read(hook_path).includes?(required_instruction) + has_current_agent_loop_instructions = ["CLAUDE.md", "AGENTS.md"].all? do |path| + agent_loop_instructions_include_guidance?(File.join(@project_root, path), required_instruction) + end + + if has_current_manifest && has_current_hook && has_current_agent_loop_instructions + DoctorCheck.new("PASS", "Generated agent instructions are current.") + else + DoctorCheck.new("FAIL", "Generated agent instructions are out of date; run `amber setup:agent`.") + end + rescue ex : JSON::ParseException + DoctorCheck.new("FAIL", "Could not parse .amber/agent_setup.json: #{ex.message}; run `amber setup:agent`.") + rescue ex : IO::Error + DoctorCheck.new("FAIL", "Could not read generated agent instructions: #{ex.message}; run `amber setup:agent`.") + end + + private def agent_loop_instructions_include_guidance?(path : String, required_instruction : String) : Bool + return false unless File.file?(path) + + content = File.read(path) + marker_start = content.index(AmberCLI::Agent::AgentSetupGuidance::DOCUMENT_START) + marker_end = content.index(AmberCLI::Agent::AgentSetupGuidance::DOCUMENT_END) + return false unless marker_start && marker_end && marker_start < marker_end + + content[marker_start...marker_end].includes?(required_instruction) + end + private def build_agent_readiness_checks : Array(DoctorCheck) hook_path = File.join(@project_root, ".amber/amber-agent-hook") unless @should_run_external_checks diff --git a/src/amber_cli/commands/setup_agent_command.cr b/src/amber_cli/commands/setup_agent_command.cr index d264f74..bf7cfd3 100644 --- a/src/amber_cli/commands/setup_agent_command.cr +++ b/src/amber_cli/commands/setup_agent_command.cr @@ -1,6 +1,7 @@ require "yaml" require "../core/base_command" require "../agent/agent_setup_manifest" +require "../agent/agent_setup_guidance" require "../agent/hook_settings" require "../agent/resolve_compiler_for_agent_loop" require "../agent/find_agent_hook_ignore_rules" @@ -12,9 +13,9 @@ module AmberCLI::Commands class SetupAgentCommand < AmberCLI::Core::BaseCommand AGENT_HOOK_SCRIPT = {{ read_file("#{__DIR__}/../templates/agent/amber-agent-hook") }} MINIMUM_AMBER_LSP_VERSION = AmberCLI::Agent::MINIMUM_AMBER_LSP_VERSION - GENERATED_HOOK_VERSION = "4" - DOCUMENT_START = "" - DOCUMENT_END = "" + GENERATED_HOOK_VERSION = AmberCLI::Agent::AgentSetupGuidance::GENERATED_HOOK_VERSION + DOCUMENT_START = AmberCLI::Agent::AgentSetupGuidance::DOCUMENT_START + DOCUMENT_END = AmberCLI::Agent::AgentSetupGuidance::DOCUMENT_END def help_description : String "Set up the Claude Code and Codex agent loop in an Amber V2 project" @@ -106,8 +107,16 @@ module AmberCLI::Commands if marker_start && marker_end && marker_start < marker_end section = content[marker_start...marker_end] updated_section = section.sub("Use `crystal spec --affected`", "Use `crystal-alpha spec --affected`") - unless updated_section.includes?("amber-lsp lookup 'Type.method'") - updated_section += "\nBefore using a library API you are not sure of, run `amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover." + updated_section = updated_section.gsub( + "Before using a library API you are not sure of, run\n`amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover.", + "", + ) + updated_section = updated_section.gsub( + "Before using a library API you are not sure of, run `amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover.", + "", + ) + unless updated_section.includes?(AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION) + updated_section += "\n#{AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION}" end unless updated_section.includes?("stop and tell the user to run `amber setup:agent`") updated_section += "\nThe SessionStart hook reports whether setup is complete. When setup is missing, stop and tell the user to run `amber setup:agent` before editing Crystal files." @@ -129,8 +138,7 @@ module AmberCLI::Commands Use `crystal-alpha spec --affected` when available. Format Crystal files with `crystal-alpha tool format`. The installed hooks hold the watcher during edits, check each changed file, and build when the agent stops. - Before using a library API you are not sure of, run - `amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover. + #{AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION} The SessionStart hook reports whether setup is complete. When setup is missing, stop and tell the user to run `amber setup:agent` before editing Crystal files. diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook index f26ce35..b1483ea 100644 --- a/src/amber_cli/templates/agent/amber-agent-hook +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -4,7 +4,7 @@ set -u project_root=$(CDPATH='' cd "$(dirname "$0")/.." && pwd -P) || exit 2 cd "$project_root" || exit 2 -HOOK_VERSION=4 +HOOK_VERSION=5 DEFAULT_MINIMUM_AMBER_LSP_VERSION=1.0.0 MAIN_FILE='__AMBER_MAIN__' @@ -636,7 +636,9 @@ case "${1:-}" in if run_setup_check; then printf '%s\n' 'Amber agent setup is OK.' printf 'amber-lsp version: %s\n' "$CHECK_VERSION" - printf '%s\n' "Before using a library API you are not sure of, run \`amber-lsp lookup 'Type.method'\` (or the LSP tool's workspaceSymbol/hover)." + cat <<'AMBER_LOOKUP_GUIDANCE' +Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints. +AMBER_LOOKUP_GUIDANCE else print_setup_refusal fi From 75eb171cffed607a2d88fb3509e08e3c43ec0b8c Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 05:05:20 -0400 Subject: [PATCH 56/69] feat(lsp): add Crystal language API card --- .github/workflows/release.yml | 7 +- README.md | 6 +- scripts/build_release.sh | 19 ++- scripts/check_crystal_card_probes.rb | 98 ++++++++++++++ spec/amber_lsp/lookup/api_cards_spec.cr | 3 +- spec/amber_lsp/lookup/hint_command_spec.cr | 19 +++ .../crystal_card_probes/01_nil_narrowing.cr | 4 + .../02_instance_variable_type.cr | 3 + .../02a_nil_argument_narrowing.cr | 4 + .../crystal_card_probes/03_spec_scope.cr | 3 + .../04_reserved_identifier.cr | 3 + .../crystal_card_probes/05_trailing_loop.cr | 5 + .../crystal_card_probes/06_named_arguments.cr | 7 + .../crystal_card_probes/07_json_require.cr | 1 + .../08_uuid_yaml_require.cr | 1 + .../09_relative_require.cr | 1 + .../10_captureless_callback.cr | 2 + .../crystal_card_probes/11_block_arity.cr | 2 + .../12_macro_type_scope.cr | 14 ++ .../13_macro_variable_scope.cr | 8 ++ .../crystal_card_probes/14_string_to_sym.cr | 2 + .../15_runtime_class_ancestry.cr | 6 + .../fixtures/crystal_card_probes/16_base64.cr | 3 + .../17_json_parse_exception.cr | 2 + .../crystal_card_probes/18_digest_api.cr | 5 + .../19_directory_creation.cr | 2 + .../crystal_card_probes/20_spec_subject.cr | 7 + .../21_integer_conversion.cr | 2 + .../22_string_concatenation.cr | 3 + .../crystal_card_probes/23_process_signal.cr | 2 + .../crystal_card_probes/24_json_any_hash.cr | 4 + .../25_json_builder_array.cr | 3 + .../26_http_headers_copy.cr | 5 + .../crystal_card_probes/27_time_nanosecond.cr | 1 + .../crystal_card_probes/28_slice_copy.cr | 2 + .../29_named_tuple_hash.cr | 1 + .../30_regex_match_capture.cr | 1 + .../31_string_byte_slice.cr | 3 + .../crystal_card_probes/32_digest_require.cr | 3 + .../33_file_utils_require.cr | 3 + .../crystal_card_probes/34_block_predicate.cr | 2 + .../crystal_card_probes/35_block_next.cr | 2 + .../crystal_card_probes/36_nil_return.cr | 3 + .../crystal_card_probes/37_checked_cast.cr | 4 + .../38_constant_assignment.cr | 1 + .../crystal_card_probes/39_block_arity.cr | 2 + .../crystal_card_probes/40_block_return.cr | 5 + .../crystal_card_probes/relative_helper.cr | 1 + src/amber_lsp/cards/crystal.yml | 127 ++++++++++++++++++ src/amber_lsp/cards/embedded_api_cards.cr | 5 +- 50 files changed, 409 insertions(+), 13 deletions(-) create mode 100755 scripts/check_crystal_card_probes.rb create mode 100644 spec/fixtures/crystal_card_probes/01_nil_narrowing.cr create mode 100644 spec/fixtures/crystal_card_probes/02_instance_variable_type.cr create mode 100644 spec/fixtures/crystal_card_probes/02a_nil_argument_narrowing.cr create mode 100644 spec/fixtures/crystal_card_probes/03_spec_scope.cr create mode 100644 spec/fixtures/crystal_card_probes/04_reserved_identifier.cr create mode 100644 spec/fixtures/crystal_card_probes/05_trailing_loop.cr create mode 100644 spec/fixtures/crystal_card_probes/06_named_arguments.cr create mode 100644 spec/fixtures/crystal_card_probes/07_json_require.cr create mode 100644 spec/fixtures/crystal_card_probes/08_uuid_yaml_require.cr create mode 100644 spec/fixtures/crystal_card_probes/09_relative_require.cr create mode 100644 spec/fixtures/crystal_card_probes/10_captureless_callback.cr create mode 100644 spec/fixtures/crystal_card_probes/11_block_arity.cr create mode 100644 spec/fixtures/crystal_card_probes/12_macro_type_scope.cr create mode 100644 spec/fixtures/crystal_card_probes/13_macro_variable_scope.cr create mode 100644 spec/fixtures/crystal_card_probes/14_string_to_sym.cr create mode 100644 spec/fixtures/crystal_card_probes/15_runtime_class_ancestry.cr create mode 100644 spec/fixtures/crystal_card_probes/16_base64.cr create mode 100644 spec/fixtures/crystal_card_probes/17_json_parse_exception.cr create mode 100644 spec/fixtures/crystal_card_probes/18_digest_api.cr create mode 100644 spec/fixtures/crystal_card_probes/19_directory_creation.cr create mode 100644 spec/fixtures/crystal_card_probes/20_spec_subject.cr create mode 100644 spec/fixtures/crystal_card_probes/21_integer_conversion.cr create mode 100644 spec/fixtures/crystal_card_probes/22_string_concatenation.cr create mode 100644 spec/fixtures/crystal_card_probes/23_process_signal.cr create mode 100644 spec/fixtures/crystal_card_probes/24_json_any_hash.cr create mode 100644 spec/fixtures/crystal_card_probes/25_json_builder_array.cr create mode 100644 spec/fixtures/crystal_card_probes/26_http_headers_copy.cr create mode 100644 spec/fixtures/crystal_card_probes/27_time_nanosecond.cr create mode 100644 spec/fixtures/crystal_card_probes/28_slice_copy.cr create mode 100644 spec/fixtures/crystal_card_probes/29_named_tuple_hash.cr create mode 100644 spec/fixtures/crystal_card_probes/30_regex_match_capture.cr create mode 100644 spec/fixtures/crystal_card_probes/31_string_byte_slice.cr create mode 100644 spec/fixtures/crystal_card_probes/32_digest_require.cr create mode 100644 spec/fixtures/crystal_card_probes/33_file_utils_require.cr create mode 100644 spec/fixtures/crystal_card_probes/34_block_predicate.cr create mode 100644 spec/fixtures/crystal_card_probes/35_block_next.cr create mode 100644 spec/fixtures/crystal_card_probes/36_nil_return.cr create mode 100644 spec/fixtures/crystal_card_probes/37_checked_cast.cr create mode 100644 spec/fixtures/crystal_card_probes/38_constant_assignment.cr create mode 100644 spec/fixtures/crystal_card_probes/39_block_arity.cr create mode 100644 spec/fixtures/crystal_card_probes/40_block_return.cr create mode 100644 spec/fixtures/crystal_card_probes/relative_helper.cr create mode 100644 src/amber_lsp/cards/crystal.yml diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index bac1458..3506014 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -103,8 +103,11 @@ jobs: - name: Create archive run: | - mkdir -p dist - tar -czf dist/amber_cli-${{ matrix.target }}.tar.gz amber amber-lsp + mkdir -p dist/package/share/amber_cli/api + cp amber amber-lsp dist/package/ + cp src/amber_lsp/cards/crystal.yml dist/package/share/amber_cli/api/crystal.yml + tar -czf dist/amber_cli-${{ matrix.target }}.tar.gz -C dist/package amber amber-lsp share + tar -tzf dist/amber_cli-${{ matrix.target }}.tar.gz | grep -F 'share/amber_cli/api/crystal.yml' - name: Calculate checksum id: checksum diff --git a/README.md b/README.md index a674c61..982d7c5 100644 --- a/README.md +++ b/README.md @@ -46,11 +46,13 @@ curl -fLO "https://github.com/amberframework/amber_cli/releases/download/${versi shasum -a 256 -c "${asset}.sha256" tar -xzf "${asset}" install -m 0755 amber amber-lsp /usr/local/bin/ +install -d -m 0755 /usr/local/share/amber_cli/api +install -m 0644 share/amber_cli/api/crystal.yml /usr/local/share/amber_cli/api/ amber --version ``` -On Linux, use `sha256sum -c` for the checksum. Prefix only the `install` -command with `sudo` if `/usr/local/bin` is not writable. +On Linux, use `sha256sum -c` for the checksum. Prefix the `install` commands +with `sudo` if `/usr/local` is not writable. ## Create and verify a web app diff --git a/scripts/build_release.sh b/scripts/build_release.sh index da6581e..019a172 100755 --- a/scripts/build_release.sh +++ b/scripts/build_release.sh @@ -21,8 +21,8 @@ ARCH=$(uname -m) case "${OS}" in "darwin") TARGET="darwin-arm64" - BUILD_CLI="crystal build src/amber_cli.cr -o amber --release" - BUILD_LSP="crystal build src/amber_lsp.cr -o amber-lsp --release" + BUILD_CLI="crystal-alpha build src/amber_cli.cr -o amber --release" + BUILD_LSP="crystal-alpha build src/amber_lsp.cr -o amber-lsp --release" CHECKSUM_CMD="shasum -a 256" if [ "${ARCH}" != "arm64" ]; then echo "⚠️ Warning: Building for ARM64 on ${ARCH} architecture" @@ -38,8 +38,8 @@ case "${OS}" in exit 1 ;; esac - BUILD_CLI="crystal build src/amber_cli.cr -o amber --release --static" - BUILD_LSP="crystal build src/amber_lsp.cr -o amber-lsp --release --static" + BUILD_CLI="crystal-alpha build src/amber_cli.cr -o amber --release --static" + BUILD_LSP="crystal-alpha build src/amber_lsp.cr -o amber-lsp --release --static" CHECKSUM_CMD="sha256sum" ;; *) @@ -53,9 +53,9 @@ echo "🎯 Building for target: ${TARGET}" # Install dependencies echo "📦 Installing dependencies..." if [ -f shard.lock ]; then - shards install --production + shards-alpha install --production else - shards install + shards-alpha install fi # Build binaries @@ -74,7 +74,12 @@ test -x amber-lsp # Create archive echo "📦 Creating archive..." -tar -czf "${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" amber amber-lsp +PACKAGE_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/amber-cli-package.XXXXXX")" +trap 'rm -rf "${PACKAGE_ROOT}"' EXIT +mkdir -p "${PACKAGE_ROOT}/share/amber_cli/api" +cp amber amber-lsp "${PACKAGE_ROOT}/" +cp src/amber_lsp/cards/crystal.yml "${PACKAGE_ROOT}/share/amber_cli/api/crystal.yml" +tar -czf "${OUTPUT_DIR}/amber_cli-${TARGET}.tar.gz" -C "${PACKAGE_ROOT}" amber amber-lsp share # Calculate checksum echo "🔢 Calculating checksum..." diff --git a/scripts/check_crystal_card_probes.rb b/scripts/check_crystal_card_probes.rb new file mode 100755 index 0000000..99af995 --- /dev/null +++ b/scripts/check_crystal_card_probes.rb @@ -0,0 +1,98 @@ +#!/usr/bin/env ruby + +require "open3" +require "tmpdir" +require "yaml" + +REPO_ROOT = File.expand_path("..", __dir__) +CARD_PATH = File.join(REPO_ROOT, "src", "amber_lsp", "cards", "crystal.yml") +PROBE_ROOT = File.join(REPO_ROOT, "spec", "fixtures", "crystal_card_probes") +PROBE_FILES = [ + "01_nil_narrowing.cr", + "02a_nil_argument_narrowing.cr", + "02_instance_variable_type.cr", + "03_spec_scope.cr", + "04_reserved_identifier.cr", + "05_trailing_loop.cr", + "06_named_arguments.cr", + "07_json_require.cr", + "08_uuid_yaml_require.cr", + "09_relative_require.cr", + "10_captureless_callback.cr", + "11_block_arity.cr", + "12_macro_type_scope.cr", + "13_macro_variable_scope.cr", + "14_string_to_sym.cr", + "15_runtime_class_ancestry.cr", + "16_base64.cr", + "17_json_parse_exception.cr", + "18_digest_api.cr", + "21_integer_conversion.cr", + "22_string_concatenation.cr", + "23_process_signal.cr", + "24_json_any_hash.cr", + "25_json_builder_array.cr", + "26_http_headers_copy.cr", + "27_time_nanosecond.cr", + "28_slice_copy.cr", + "29_named_tuple_hash.cr", + "30_regex_match_capture.cr", + "31_string_byte_slice.cr", + "32_digest_require.cr", + "33_file_utils_require.cr", + "34_block_predicate.cr", + "35_block_next.cr", + "19_directory_creation.cr", + "20_spec_subject.cr", + "36_nil_return.cr", + "37_checked_cast.cr", + "38_constant_assignment.cr", + "39_block_arity.cr", + "40_block_return.cr", +].freeze + +def normalize_code(source) + source.gsub(/;/, " ").gsub(/\s+/, " ").strip +end + +def compile_card_probes + card = YAML.load_file(CARD_PATH) + hints = card.fetch("error_hints") + raise "#{hints.size} card hints do not match #{PROBE_FILES.size} probe fixtures" unless hints.size == PROBE_FILES.size + + failures = [] + Dir.mktmpdir("amber-lsp-crystal-card-cache") do |cache_root| + hints.zip(PROBE_FILES).each do |hint, probe_file| + probe_path = File.join(PROBE_ROOT, probe_file) + source = File.read(probe_path) + example = hint.fetch("example") + unless normalize_code(source).include?(normalize_code(example)) + failures << "#{probe_file}: card example is not present in the probe source" + next + end + + stdout, stderr, status = Open3.capture3( + {"CRYSTAL_CACHE_DIR" => cache_root}, + "crystal-alpha", + "build", + "--no-codegen", + probe_path, + chdir: REPO_ROOT, + ) + next if status.success? && stdout.empty? && stderr.empty? + + failures << "#{probe_file}: #{(stderr + stdout).strip}" + end + end + + if failures.empty? + puts "GREEN: #{PROBE_FILES.size}/#{PROBE_FILES.size} Crystal card examples compiled with zero warnings." + return true + end + + failures.each { |failure| warn failure } + warn "RED: #{PROBE_FILES.size - failures.size}/#{PROBE_FILES.size} Crystal card probes passed." + false +end + +exit(compile_card_probes ? 0 : 1) diff --git a/spec/amber_lsp/lookup/api_cards_spec.cr b/spec/amber_lsp/lookup/api_cards_spec.cr index e1f6bdb..2428c2d 100644 --- a/spec/amber_lsp/lookup/api_cards_spec.cr +++ b/spec/amber_lsp/lookup/api_cards_spec.cr @@ -16,10 +16,11 @@ describe AmberLSP::Lookup::LoadAPICards do cards = AmberLSP::Lookup::LoadAPICards.new(project).perform cards.list_of_errors.should be_empty - cards.list_of_cards.map(&.card.library).should eq(["fixture_project", "example"]) + cards.list_of_cards.map(&.card.library).should eq(["fixture_project", "example", "crystal"]) cards.docs_flags_by_library.should eq({ "fixture_project" => ["fixture_project_docs"], "example" => ["example_docs"], + "crystal" => [] of String, }) cards.docs_entries_by_library.should eq({ "example" => ["src/example.cr", "src/ui.cr"], diff --git a/spec/amber_lsp/lookup/hint_command_spec.cr b/spec/amber_lsp/lookup/hint_command_spec.cr index 1d07bd7..b6267fe 100644 --- a/spec/amber_lsp/lookup/hint_command_spec.cr +++ b/spec/amber_lsp/lookup/hint_command_spec.cr @@ -75,6 +75,25 @@ describe AmberLSP::Lookup::RunHintCommand do output.to_s.should contain("Usage: amber-lsp hint [--root DIR]") output.to_s.should contain("compiler output from stdin") end + + it "uses the compiled Crystal language card for a plain project" do + with_tempdir do |project| + input = IO::Memory.new("undefined method 'to_sym' for String\n") + output = IO::Memory.new + code = AmberLSP::Lookup::RunHintCommand.new( + ["--root", project], + input, + output, + IO::Memory.new, + ).perform + + code.should eq(0) + output.to_s.lines.should eq([ + "hint: Crystal String has no `to_sym`; keep values as String, or call `to_s` when converting a Symbol to String. [crystal@1.21.0]", + " right: name.to_s", + ]) + end + end end private def with_hint_cli_project(&) diff --git a/spec/fixtures/crystal_card_probes/01_nil_narrowing.cr b/spec/fixtures/crystal_card_probes/01_nil_narrowing.cr new file mode 100644 index 0000000..64e95c5 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/01_nil_narrowing.cr @@ -0,0 +1,4 @@ +optional_value : String? = "record" +if value = optional_value + value.size +end diff --git a/spec/fixtures/crystal_card_probes/02_instance_variable_type.cr b/spec/fixtures/crystal_card_probes/02_instance_variable_type.cr new file mode 100644 index 0000000..bfbd357 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/02_instance_variable_type.cr @@ -0,0 +1,3 @@ +class CardProbePosts + @posts : Array(String) = [] of String +end diff --git a/spec/fixtures/crystal_card_probes/02a_nil_argument_narrowing.cr b/spec/fixtures/crystal_card_probes/02a_nil_argument_narrowing.cr new file mode 100644 index 0000000..fa8d079 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/02a_nil_argument_narrowing.cr @@ -0,0 +1,4 @@ +optional_value : String? = "ready" +if value = optional_value + value.size +end diff --git a/spec/fixtures/crystal_card_probes/03_spec_scope.cr b/spec/fixtures/crystal_card_probes/03_spec_scope.cr new file mode 100644 index 0000000..84fff99 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/03_spec_scope.cr @@ -0,0 +1,3 @@ +private def helper : String + "ready" +end diff --git a/spec/fixtures/crystal_card_probes/04_reserved_identifier.cr b/spec/fixtures/crystal_card_probes/04_reserved_identifier.cr new file mode 100644 index 0000000..924b548 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/04_reserved_identifier.cr @@ -0,0 +1,3 @@ +class Invoice; end + +def configure(macro_name : Symbol? = nil, protected_environments : Array(String) = [] of String); end diff --git a/spec/fixtures/crystal_card_probes/05_trailing_loop.cr b/spec/fixtures/crystal_card_probes/05_trailing_loop.cr new file mode 100644 index 0000000..b22a97e --- /dev/null +++ b/spec/fixtures/crystal_card_probes/05_trailing_loop.cr @@ -0,0 +1,5 @@ +values = [1, 2] +index = 0 +while index < values.size + index += 1 +end diff --git a/spec/fixtures/crystal_card_probes/06_named_arguments.cr b/spec/fixtures/crystal_card_probes/06_named_arguments.cr new file mode 100644 index 0000000..6ed0d40 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/06_named_arguments.cr @@ -0,0 +1,7 @@ +def foo(a : Int32, b : Int32) : Int32 + a + b +end + +a = 1 +options = {b: 2} +foo(a: a, b: options[:b]) diff --git a/spec/fixtures/crystal_card_probes/07_json_require.cr b/spec/fixtures/crystal_card_probes/07_json_require.cr new file mode 100644 index 0000000..b2112af --- /dev/null +++ b/spec/fixtures/crystal_card_probes/07_json_require.cr @@ -0,0 +1 @@ +require "json" diff --git a/spec/fixtures/crystal_card_probes/08_uuid_yaml_require.cr b/spec/fixtures/crystal_card_probes/08_uuid_yaml_require.cr new file mode 100644 index 0000000..2463cb2 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/08_uuid_yaml_require.cr @@ -0,0 +1 @@ +require "uuid/yaml" diff --git a/spec/fixtures/crystal_card_probes/09_relative_require.cr b/spec/fixtures/crystal_card_probes/09_relative_require.cr new file mode 100644 index 0000000..aa19bba --- /dev/null +++ b/spec/fixtures/crystal_card_probes/09_relative_require.cr @@ -0,0 +1 @@ +require "./relative_helper" diff --git a/spec/fixtures/crystal_card_probes/10_captureless_callback.cr b/spec/fixtures/crystal_card_probes/10_captureless_callback.cr new file mode 100644 index 0000000..73096bb --- /dev/null +++ b/spec/fixtures/crystal_card_probes/10_captureless_callback.cr @@ -0,0 +1,2 @@ +callback = -> { true } +callback.call diff --git a/spec/fixtures/crystal_card_probes/11_block_arity.cr b/spec/fixtures/crystal_card_probes/11_block_arity.cr new file mode 100644 index 0000000..a00a383 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/11_block_arity.cr @@ -0,0 +1,2 @@ +require "json" +JSON.build { |json| json.object { json.field "mode", "none" } } diff --git a/spec/fixtures/crystal_card_probes/12_macro_type_scope.cr b/spec/fixtures/crystal_card_probes/12_macro_type_scope.cr new file mode 100644 index 0000000..c369e75 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/12_macro_type_scope.cr @@ -0,0 +1,14 @@ +class CardProbeFields + @value : String + + macro first_ivar_name + {{ @type.instance_vars.first.name.stringify }}; end + + def initialize(@value : String); end + + def name : String + first_ivar_name + end +end + +CardProbeFields.new("value").name diff --git a/spec/fixtures/crystal_card_probes/13_macro_variable_scope.cr b/spec/fixtures/crystal_card_probes/13_macro_variable_scope.cr new file mode 100644 index 0000000..87000d3 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/13_macro_variable_scope.cr @@ -0,0 +1,8 @@ +macro define_value_method + {% method_name = "value".id %}; def {{method_name}} : String; "ready"; end; end + +class CardProbeMacro + define_value_method +end + +CardProbeMacro.new.value diff --git a/spec/fixtures/crystal_card_probes/14_string_to_sym.cr b/spec/fixtures/crystal_card_probes/14_string_to_sym.cr new file mode 100644 index 0000000..3c43d8e --- /dev/null +++ b/spec/fixtures/crystal_card_probes/14_string_to_sym.cr @@ -0,0 +1,2 @@ +name : Symbol = :name +name.to_s diff --git a/spec/fixtures/crystal_card_probes/15_runtime_class_ancestry.cr b/spec/fixtures/crystal_card_probes/15_runtime_class_ancestry.cr new file mode 100644 index 0000000..76f5b6c --- /dev/null +++ b/spec/fixtures/crystal_card_probes/15_runtime_class_ancestry.cr @@ -0,0 +1,6 @@ +class CardProbeBase; end + +class CardProbeChild < CardProbeBase; end + +instance = CardProbeChild.new +instance.is_a?(CardProbeBase) diff --git a/spec/fixtures/crystal_card_probes/16_base64.cr b/spec/fixtures/crystal_card_probes/16_base64.cr new file mode 100644 index 0000000..3aac8c4 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/16_base64.cr @@ -0,0 +1,3 @@ +require "base64" +token = "cGF5bG9hZA==" +Base64.decode(token) diff --git a/spec/fixtures/crystal_card_probes/17_json_parse_exception.cr b/spec/fixtures/crystal_card_probes/17_json_parse_exception.cr new file mode 100644 index 0000000..5a6f761 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/17_json_parse_exception.cr @@ -0,0 +1,2 @@ +require "json" +JSON::ParseException.new("invalid JSON", 1, 1) diff --git a/spec/fixtures/crystal_card_probes/18_digest_api.cr b/spec/fixtures/crystal_card_probes/18_digest_api.cr new file mode 100644 index 0000000..123de13 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/18_digest_api.cr @@ -0,0 +1,5 @@ +require "digest/sha256" +path = "input.txt" +digest = Digest::SHA256.new +digest.file(path) +digest.hexfinal diff --git a/spec/fixtures/crystal_card_probes/19_directory_creation.cr b/spec/fixtures/crystal_card_probes/19_directory_creation.cr new file mode 100644 index 0000000..9ef49ed --- /dev/null +++ b/spec/fixtures/crystal_card_probes/19_directory_creation.cr @@ -0,0 +1,2 @@ +path = "tmp/card-probe" +Dir.mkdir_p(path) diff --git a/spec/fixtures/crystal_card_probes/20_spec_subject.cr b/spec/fixtures/crystal_card_probes/20_spec_subject.cr new file mode 100644 index 0000000..8467c0e --- /dev/null +++ b/spec/fixtures/crystal_card_probes/20_spec_subject.cr @@ -0,0 +1,7 @@ +class CardProbeSubject + def self.value : Int32 + 1 + end +end + +CardProbeSubject.value diff --git a/spec/fixtures/crystal_card_probes/21_integer_conversion.cr b/spec/fixtures/crystal_card_probes/21_integer_conversion.cr new file mode 100644 index 0000000..de0617a --- /dev/null +++ b/spec/fixtures/crystal_card_probes/21_integer_conversion.cr @@ -0,0 +1,2 @@ +value = 42_i32 +value.to_i64 diff --git a/spec/fixtures/crystal_card_probes/22_string_concatenation.cr b/spec/fixtures/crystal_card_probes/22_string_concatenation.cr new file mode 100644 index 0000000..726019b --- /dev/null +++ b/spec/fixtures/crystal_card_probes/22_string_concatenation.cr @@ -0,0 +1,3 @@ +text = "title" +text = text + "x" +text diff --git a/spec/fixtures/crystal_card_probes/23_process_signal.cr b/spec/fixtures/crystal_card_probes/23_process_signal.cr new file mode 100644 index 0000000..284f432 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/23_process_signal.cr @@ -0,0 +1,2 @@ +process = Process.new("true") +Process.signal(Signal::KILL, process.pid) diff --git a/spec/fixtures/crystal_card_probes/24_json_any_hash.cr b/spec/fixtures/crystal_card_probes/24_json_any_hash.cr new file mode 100644 index 0000000..2f949f3 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/24_json_any_hash.cr @@ -0,0 +1,4 @@ +require "json" + +values = JSON.parse("{}").as_h +values["mode"] = JSON::Any.new("ready") diff --git a/spec/fixtures/crystal_card_probes/25_json_builder_array.cr b/spec/fixtures/crystal_card_probes/25_json_builder_array.cr new file mode 100644 index 0000000..c0fc1d9 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/25_json_builder_array.cr @@ -0,0 +1,3 @@ +require "json" + +JSON.build { |json| json.array { json.string("ready") } } diff --git a/spec/fixtures/crystal_card_probes/26_http_headers_copy.cr b/spec/fixtures/crystal_card_probes/26_http_headers_copy.cr new file mode 100644 index 0000000..bed8fc4 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/26_http_headers_copy.cr @@ -0,0 +1,5 @@ +require "http/headers" + +headers = {"content-type" => "application/json"} +http_headers = HTTP::Headers.new +headers.each { |key, value| http_headers[key] = value } diff --git a/spec/fixtures/crystal_card_probes/27_time_nanosecond.cr b/spec/fixtures/crystal_card_probes/27_time_nanosecond.cr new file mode 100644 index 0000000..8027e1a --- /dev/null +++ b/spec/fixtures/crystal_card_probes/27_time_nanosecond.cr @@ -0,0 +1 @@ +Time.utc(2026, 1, 1, 0, 0, 0, nanosecond: 1) diff --git a/spec/fixtures/crystal_card_probes/28_slice_copy.cr b/spec/fixtures/crystal_card_probes/28_slice_copy.cr new file mode 100644 index 0000000..d3c9b35 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/28_slice_copy.cr @@ -0,0 +1,2 @@ +body = Slice[1_u8, 2_u8] +body.dup diff --git a/spec/fixtures/crystal_card_probes/29_named_tuple_hash.cr b/spec/fixtures/crystal_card_probes/29_named_tuple_hash.cr new file mode 100644 index 0000000..2145fc0 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/29_named_tuple_hash.cr @@ -0,0 +1 @@ +values = {id: 1_i64}.to_h.all? { |key, value| key.to_s == "id" && value > 0 } diff --git a/spec/fixtures/crystal_card_probes/30_regex_match_capture.cr b/spec/fixtures/crystal_card_probes/30_regex_match_capture.cr new file mode 100644 index 0000000..bd726f3 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/30_regex_match_capture.cr @@ -0,0 +1 @@ +"a".scan(/a/).map(&.[0]) diff --git a/spec/fixtures/crystal_card_probes/31_string_byte_slice.cr b/spec/fixtures/crystal_card_probes/31_string_byte_slice.cr new file mode 100644 index 0000000..2154370 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/31_string_byte_slice.cr @@ -0,0 +1,3 @@ +text = "prefix:value" +prefix = "prefix:" +text.byte_slice(prefix.bytesize) diff --git a/spec/fixtures/crystal_card_probes/32_digest_require.cr b/spec/fixtures/crystal_card_probes/32_digest_require.cr new file mode 100644 index 0000000..469e42c --- /dev/null +++ b/spec/fixtures/crystal_card_probes/32_digest_require.cr @@ -0,0 +1,3 @@ +require "digest/sha256" + +Digest::SHA256.hexdigest("ready") diff --git a/spec/fixtures/crystal_card_probes/33_file_utils_require.cr b/spec/fixtures/crystal_card_probes/33_file_utils_require.cr new file mode 100644 index 0000000..22cd277 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/33_file_utils_require.cr @@ -0,0 +1,3 @@ +require "file_utils" + +FileUtils.mkdir_p("/tmp/crystal-card-probe") diff --git a/spec/fixtures/crystal_card_probes/34_block_predicate.cr b/spec/fixtures/crystal_card_probes/34_block_predicate.cr new file mode 100644 index 0000000..3621c56 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/34_block_predicate.cr @@ -0,0 +1,2 @@ +queries = ["SELECT 1", "INSERT INTO things"] +queries.count { |query| query.includes?("SELECT") || query.includes?("INSERT") } diff --git a/spec/fixtures/crystal_card_probes/35_block_next.cr b/spec/fixtures/crystal_card_probes/35_block_next.cr new file mode 100644 index 0000000..2a47ad3 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/35_block_next.cr @@ -0,0 +1,2 @@ +items = ["ready", nil] +items.each { |item| next if item.nil? } diff --git a/spec/fixtures/crystal_card_probes/36_nil_return.cr b/spec/fixtures/crystal_card_probes/36_nil_return.cr new file mode 100644 index 0000000..8ea5339 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/36_nil_return.cr @@ -0,0 +1,3 @@ +optional_value : String? = "ready" +value = optional_value || raise "missing" +value.size diff --git a/spec/fixtures/crystal_card_probes/37_checked_cast.cr b/spec/fixtures/crystal_card_probes/37_checked_cast.cr new file mode 100644 index 0000000..81770ee --- /dev/null +++ b/spec/fixtures/crystal_card_probes/37_checked_cast.cr @@ -0,0 +1,4 @@ +value : Array(String) | String = [] of String +if value.is_a?(Array(String)) + value +end diff --git a/spec/fixtures/crystal_card_probes/38_constant_assignment.cr b/spec/fixtures/crystal_card_probes/38_constant_assignment.cr new file mode 100644 index 0000000..7f3c0d5 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/38_constant_assignment.cr @@ -0,0 +1 @@ +API_LEVEL = 1 diff --git a/spec/fixtures/crystal_card_probes/39_block_arity.cr b/spec/fixtures/crystal_card_probes/39_block_arity.cr new file mode 100644 index 0000000..c692642 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/39_block_arity.cr @@ -0,0 +1,2 @@ +values = [1, 2] +values.each { } diff --git a/spec/fixtures/crystal_card_probes/40_block_return.cr b/spec/fixtures/crystal_card_probes/40_block_return.cr new file mode 100644 index 0000000..6b2b082 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/40_block_return.cr @@ -0,0 +1,5 @@ +def with_integer(&block : -> Int64) : Int64 + block.call +end + +with_integer { 1_i64 } diff --git a/spec/fixtures/crystal_card_probes/relative_helper.cr b/spec/fixtures/crystal_card_probes/relative_helper.cr new file mode 100644 index 0000000..9a6c750 --- /dev/null +++ b/spec/fixtures/crystal_card_probes/relative_helper.cr @@ -0,0 +1 @@ +CARD_PROBE_HELPER = 1 diff --git a/src/amber_lsp/cards/crystal.yml b/src/amber_lsp/cards/crystal.yml new file mode 100644 index 0000000..0273bd5 --- /dev/null +++ b/src/amber_lsp/cards/crystal.yml @@ -0,0 +1,127 @@ +card_version: 1 +library: crystal +applies_to: ">= 1.21.0, < 2.0.0" +error_hints: + - pattern: "undefined method '([^']+)' for [^(]+ \\(compile-time type is \\((.+ \\| Nil)\\)\\)" + hint: 'Narrow the `{2}` value before calling `{1}`; use an `if` expression or remove nil entries when that fits the collection.' + example: 'if value = optional_value; value.size; end' + - pattern: "expected argument #\\d+ to '([^']+)' .*not \\([^\\n]* \\| Nil\\)" + hint: 'Narrow the optional argument passed to `{1}` before the call; an `if` binding keeps the non-nil value in scope.' + example: 'if value = optional_value; value.size; end' + - pattern: "(?|can't infer the type of instance variable '@([A-Za-z_][A-Za-z0-9_]*)'|this 'initialize' doesn't explicitly initialize instance variable '@([A-Za-z_][A-Za-z0-9_]*)' of)" + hint: 'Crystal needs a declaration or initialization for `@{1}`. Add a type restriction or assign it on every initialize path.' + example: '@posts : Array(String) = [] of String' + - pattern: "can't declare (def|constant|class) dynamically" + hint: 'Crystal cannot declare a `{1}` inside a runtime `describe` or `it` block; define it at file or module scope.' + example: 'private def helper : String; "ready"; end' + - pattern: "(?|cannot use '([^']+)' as a (?:parameter|local variable) name|([^ ]+) is not a class, it's a annotation)" + hint: 'The name `{1}` is reserved by Crystal; rename it to an ordinary identifier.' + example: 'class Invoice; end; def configure(macro_name : Symbol? = nil, protected_environments : Array(String) = [] of String); end' + - pattern: "trailing `(while|until)` is not supported" + hint: 'Crystal does not support a trailing `{1}` modifier; put the condition before the loop body.' + example: 'while index < values.size; index += 1; end' + - pattern: "expected named argument, not (\\*\\*)" + hint: 'Ruby-style `{1}options` forwarding is unavailable here; pass the supported named arguments explicitly.' + example: 'foo(a: a, b: options[:b])' + - pattern: "undefined constant (JSON::Serializable)" + hint: 'Load `{1}` from Crystal’s JSON standard library with `require "json"`.' + example: 'require "json"' + - pattern: "expected argument #1 to '(UUID\\.new)' .*not YAML::ParseContext" + hint: 'The YAML adapter for `{1}` is loaded with `require "uuid/yaml"`.' + example: 'require "uuid/yaml"' + - pattern: "can't find file '([^']+)'" + hint: 'Could not resolve `{1}`. Resolve local requires from the source file with `./` or the needed number of `../` segments.' + example: 'require "./relative_helper"' + - pattern: "can't send closure to C function \\(closured vars: ([^)]+)\\)" + hint: 'The C callback captures `{1}`. Move that state to stable storage and pass a captureless callback.' + example: 'callback = -> { true }; callback.call' + - pattern: "wrong number of block parameters \\(given (\\d+), expected (\\d+)\\)" + hint: 'The block receives {1} parameters but this API expects {2}; use the block arity defined by that call.' + example: 'json.object { json.field "mode", "none" }' + - pattern: "TypeNode#(instance_vars|ancestors|superclass)`? cannot be called in the top-level scope" + hint: 'Move `{1}` into a macro expanded in the target type’s scope.' + example: 'class CardProbeFields; @value : String; macro first_ivar_name; {{ @type.instance_vars.first.name.stringify }}; end; def initialize(@value : String); end; def name : String; first_ivar_name; end; end; CardProbeFields.new("value").name' + - pattern: "undefined macro variable '([^']+)'" + hint: 'Macro variable `{1}` must be declared and used inside the same macro expansion scope.' + example: 'macro define_value_method; {% method_name = "value".id %}; def {{method_name}} : String; "ready"; end; end; class CardProbeMacro; define_value_method; end; CardProbeMacro.new.value' + - pattern: "undefined method '(to_sym)' for String" + hint: 'Crystal String has no `{1}`; keep values as String, or call `to_s` when converting a Symbol to String.' + example: 'name.to_s' + - pattern: "undefined method '(ancestors|superclass)' for ([A-Za-z_:][A-Za-z0-9_:]*)\\.class" + hint: 'Runtime class objects do not expose `{1}` for `{2}`; use `is_a?` for runtime checks or a macro for compile-time ancestry.' + example: 'instance.is_a?(CardProbeBase)' + - pattern: "(?|undefined method '([^']+)' for Base64:Module|undefined constant Base64::Decoder::([^\\s]+))" + hint: 'Base64 does not expose `{1}`; use the supported `Base64.decode` or `Base64.decode_string` method for URL-safe input.' + example: 'Base64.decode(token)' + - pattern: "wrong number of arguments for '(JSON::ParseException\\.new)' \\(given 1, expected 3\\.\\.4\\)" + hint: '`{1}` needs parser location arguments; use `ArgumentError` for request errors that are not JSON parse failures.' + example: 'JSON::ParseException.new("invalid JSON", 1, 1)' + - pattern: "undefined method '(file|hexdigest)' for Digest::SHA256(?:\\.class)?" + hint: 'Digest::SHA256 `{1}` is used on an instance for file/stream operations; call `hexfinal` for a streaming digest.' + example: 'digest = Digest::SHA256.new; digest.file(path); digest.hexfinal' + - pattern: "undefined method '(to_i64\\?)' for Int32" + hint: 'Int32 provides `{1}` for a checked conversion; use `to_i64` when the value is known to fit.' + example: 'value.to_i64' + - pattern: "undefined method '(<<)' for String" + hint: 'Crystal String does not support `{1}`; assign the concatenated String returned by `+`.' + example: 'text = text + "x"' + - pattern: "undefined method '(kill)' for Process" + hint: 'Use `{1}` as a Process instance method or send a signal with `Process.signal`.' + example: 'Process.signal(Signal::KILL, process.pid)' + - pattern: "undefined method '([^']+)' for JSON::Any" + hint: 'JSON::Any exposes a parsed object through `{1}`; mutate its Hash returned by `as_h`.' + example: 'values = JSON.parse("{}").as_h; values["mode"] = JSON::Any.new("ready")' + - pattern: "'JSON::Builder#(array)' is expected to be invoked with a block" + hint: '`{1}` needs a block that writes each JSON value.' + example: 'JSON.build { |json| json.array { json.string("ready") } }' + - pattern: "wrong number of arguments for '(HTTP::Headers.new)' \\(given 1, expected 0\\)" + hint: '`{1}` takes no initial Hash; create it empty and copy each header pair.' + example: 'http_headers = HTTP::Headers.new; headers.each { |key, value| http_headers[key] = value }' + - pattern: "wrong number of arguments for '(Time.utc)' \\(given 7, expected 0, 3, 4, 5, 6\\)" + hint: '`{1}` takes nanoseconds as a named argument after the seconds.' + example: 'Time.utc(2026, 1, 1, 0, 0, 0, nanosecond: 1)' + - pattern: "no overload matches '(Slice\\(UInt8\\).new)' with type Slice\\(UInt8\\)" + hint: '`{1}` copies an existing byte slice with `dup`.' + example: 'body.dup' + - pattern: "undefined method '(all\\?)' for NamedTuple" + hint: 'NamedTuple does not provide `{1}`; convert it to a Hash first.' + example: 'values = {id: 1_i64}.to_h.all? { |key, value| key.to_s == "id" && value > 0 }' + - pattern: "undefined method '(matched)' for Regex::MatchData" + hint: 'Use `{1}` on the match result tuple to read the captured text.' + example: '"a".scan(/a/).map(&.[0])' + - pattern: "undefined method '(delete_prefix)' for String" + hint: 'String does not provide `{1}`; use `byte_slice` with the prefix byte count.' + example: 'text.byte_slice(prefix.bytesize)' + - pattern: "undefined constant (Digest::SHA256)" + hint: 'Load `{1}` from the Crystal standard library with `require "digest/sha256"`.' + example: 'require "digest/sha256"' + - pattern: "undefined constant (FileUtils)" + hint: 'Load `{1}` from the Crystal standard library with `require "file_utils"`.' + example: 'require "file_utils"' + - pattern: "expecting token '\\)', not '([|][|])'" + hint: 'This block shorthand cannot combine predicates with `{1}`; name the block value and call both methods.' + example: 'queries.count { |query| query.includes?("SELECT") || query.includes?("INSERT") }' + - pattern: "can't (return) from captured block, use next" + hint: 'A captured block cannot use `{1}` to return from its enclosing method; use `next` to skip this block iteration.' + example: 'items.each { |item| next if item.nil? }' + - pattern: "undefined method '(mkdir_p)' for File:Module" + hint: 'Directory creation uses `Dir.{1}`; `File` handles individual files.' + example: 'Dir.mkdir_p(path)' + - pattern: "undefined local variable or method '(described_class)' for top-level" + hint: 'Crystal Spec does not provide `{1}`; name the type under test or assign it to a local constant.' + example: 'CardProbeSubject.value' + - pattern: "method ([^ ]+) must return (.+) but it is returning \\((.+ \\| Nil)\\)" + hint: 'Method `{1}` must return `{2}`; supply a fallback or raise when its expression is nil.' + example: 'value = optional_value || raise "missing"; value.size' + - pattern: "can't cast (.+) to \\((.+)\\)" + hint: 'The inferred value `{1}` cannot be cast to `{2}`; check its concrete type with `is_a?` before using it.' + example: 'if value.is_a?(Array(String)); value; end' + - pattern: "expecting token '(:)', not '(=)'" + hint: 'Crystal expected `{1}` before `{2}`; constants infer their type from `NAME = value`, while variable types use `name : Type`.' + example: 'API_LEVEL = 1' + - pattern: "too many block parameters \\(given (\\d+), expected maximum (\\d+)\\)" + hint: 'The block receives `{1}` parameters but this API allows at most `{2}`; use the arity expected by the call.' + example: 'values.each { }' + - pattern: "expected block to return (.+), not (.+)" + hint: 'This block must return `{1}`; make its final expression a value accepted by that type.' + example: 'with_integer { 1_i64 }' diff --git a/src/amber_lsp/cards/embedded_api_cards.cr b/src/amber_lsp/cards/embedded_api_cards.cr index 3003188..f2625c6 100644 --- a/src/amber_lsp/cards/embedded_api_cards.cr +++ b/src/amber_lsp/cards/embedded_api_cards.cr @@ -2,5 +2,8 @@ module AmberLSP::Lookup alias EmbeddedAPICardYAML = Tuple(String, String) # :nodoc: - EMBEDDED_API_CARD_YAMLS = [] of EmbeddedAPICardYAML + EMBEDDED_API_CARD_YAMLS = [{ + "crystal.yml", + {{ read_file("src/amber_lsp/cards/crystal.yml") }}, + }] end From a58715a0c4cf0038267336d7c5d1db0e4e9fa5f9 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 05:13:10 -0400 Subject: [PATCH 57/69] feat(lsp): add Crystal card evaluation --- scripts/crystal_card_eval.rb | 586 +++++++++++++++++++++++++++++++++++ 1 file changed, 586 insertions(+) create mode 100755 scripts/crystal_card_eval.rb diff --git a/scripts/crystal_card_eval.rb b/scripts/crystal_card_eval.rb new file mode 100755 index 0000000..aee99c7 --- /dev/null +++ b/scripts/crystal_card_eval.rb @@ -0,0 +1,586 @@ +#!/usr/bin/env ruby + +require "digest" +require "fileutils" +require "json" +require "open3" +require "optparse" +require "tmpdir" +require "time" +require "yaml" + +REPO_ROOT = File.expand_path("..", __dir__) +MINING_ROOT = "/Users/crimsonknight/agent_transcript_archive/mining/2026-10-04" +DEFAULT_CASES_PATH = File.join(MINING_ROOT, "cases_labeled.jsonl") +DEFAULT_LABELS_PATH = File.join(MINING_ROOT, "luna_labels") +DEFAULT_BINARY_PATH = File.join(REPO_ROOT, "bin", "amber-lsp") +DEFAULT_REPORT_PATH = File.join(MINING_ROOT, "eval", "CRYSTAL_CARD_EVAL_REPORT.md") +CARD_PATH = File.join(REPO_ROOT, "src", "amber_lsp", "cards", "crystal.yml") +CATEGORY_TOTALS = { + "crystal_nil_union" => 130, + "crystal_type_inference" => 109, + "crystal_syntax" => 96, + "crystal_macro" => 71, + "crystal_stdlib_api" => 59, + "require_or_load" => 37, + "crystal_block_proc" => 23, + "spec_framework" => 18, +}.freeze +EXPECTED_CASE_COUNT = CATEGORY_TOTALS.values.inject(0, :+) + +def parse_options + options = { + :split => "authoring", + :cases_path => DEFAULT_CASES_PATH, + :labels_path => DEFAULT_LABELS_PATH, + :binary_path => DEFAULT_BINARY_PATH, + :report_path => nil, + :heldout_label => nil, + :show_misses => false, + } + + parser = OptionParser.new do |options_parser| + options_parser.banner = "Usage: ruby scripts/crystal_card_eval.rb [options]" + options_parser.on("--split SPLIT", "authoring, heldout, or both (default: authoring)") do |value| + options[:split] = value + end + options_parser.on("--heldout-label LABEL", "Required for heldout measurements; recorded in the report") do |value| + options[:heldout_label] = value + end + options_parser.on("--cases PATH", "Path to cases_labeled.jsonl") do |value| + options[:cases_path] = value + end + options_parser.on("--labels DIR", "Directory containing the Luna label JSONL files") do |value| + options[:labels_path] = value + end + options_parser.on("--binary PATH", "amber-lsp executable to evaluate") do |value| + options[:binary_path] = value + end + options_parser.on("--report PATH", "Write the final both-split report to PATH") do |value| + options[:report_path] = value + end + options_parser.on("--show-misses", "Print each miss after the category summary") do + options[:show_misses] = true + end + options_parser.on("-h", "--help", "Show this help") do + puts options_parser + exit 0 + end + end + + parser.parse! + unless %w[authoring heldout both].include?(options[:split]) + raise OptionParser::InvalidArgument, "--split must be authoring, heldout, or both" + end + if options[:split] != "authoring" && options[:heldout_label].to_s.empty? + raise OptionParser::MissingArgument, "--heldout-label is required for heldout evaluation" + end + if options[:report_path] && options[:split] != "both" + raise OptionParser::InvalidArgument, "--report is written only by --split both" + end + options[:report_path] ||= DEFAULT_REPORT_PATH if options[:split] == "both" + options +end + +def read_case_records(path) + list_of_records = {} + File.foreach(path) do |line| + next if line.strip.empty? + + payload = JSON.parse(line) + list_of_records[payload.fetch("case_id")] = payload + end + list_of_records +end + +def trainable_labeled_cases(labels_path, case_records) + list_of_cases = [] + seen_case_ids = {} + counts_by_category = Hash.new(0) + + Dir.glob(File.join(labels_path, "*.jsonl")).sort.each do |label_path| + File.foreach(label_path) do |line| + next if line.strip.empty? + + label = JSON.parse(line) + category = label["category"] + next unless CATEGORY_TOTALS.has_key?(category) + next unless label["trainable"] == "yes" + + case_id = label.fetch("case_id") + raise "duplicate trainable label for #{case_id}" if seen_case_ids.has_key?(case_id) + + numeric_suffix = case_id.match(/(\d+)\z/) + raise "case_id has no numeric suffix: #{case_id}" unless numeric_suffix + + case_record = case_records[case_id] + raise "no case record joined for #{case_id}" unless case_record + + split = numeric_suffix[1].to_i.even? ? "authoring" : "heldout" + list_of_cases << { + :case_id => case_id, + :category => category, + :split => split, + :wrong => label["wrong"].to_s, + :right => label["right"].to_s, + :error => case_record["error"].to_s, + :compiler_snippet => case_record["compiler_snippet"].to_s, + } + seen_case_ids[case_id] = true + counts_by_category[category] += 1 + end + end + + unless list_of_cases.size == EXPECTED_CASE_COUNT + raise "expected #{EXPECTED_CASE_COUNT} trainable cases, joined #{list_of_cases.size}" + end + CATEGORY_TOTALS.each do |category, expected_count| + actual_count = counts_by_category[category] + raise "expected #{expected_count} #{category} cases, joined #{actual_count}" unless actual_count == expected_count + end + + list_of_cases.sort_by { |item| item[:case_id] } +end + +def parse_hint_output(output) + list_of_hints = [] + current_hint = nil + output.each_line do |raw_line| + line = raw_line.chomp + if match = line.match(/\Ahint: (.*) \[([^@\]]+)@([^\]]+)\]\z/) + current_hint = { + :text => match[1], + :library => match[2], + :version => match[3], + :example => "", + } + list_of_hints << current_hint + elsif line.start_with?(" right: ") && current_hint + current_hint[:example] = line.sub(/\A right: /, "").strip + end + end + list_of_hints +end + +def normalize_code(source) + source.to_s + .gsub("`", "") + .gsub(/\r\n?/, "\n") + .gsub(/\s+/, " ") + .strip +end + +def right_form_match_rule(item, hint) + expected = normalize_code(item[:right]) + suggested = normalize_code(hint[:example]) + return "normalized equality" if expected == suggested + if expected.size > 3 && suggested.size > 3 && (expected.include?(suggested) || suggested.include?(expected)) + return "containment" + end + + wrong = normalize_code(item[:wrong]) + error = normalize_code(item[:error]) + hint_text = hint[:text] + + nilable_diagnostic = error.include?("compile-time type is") && error.include?("| Nil") + nilable_argument = error.include?("expected argument") && error.include?("| Nil") + nilable_return = error.include?("must return") && error.include?("| Nil") + if nilable_diagnostic || nilable_argument || nilable_return + if expected.start_with?("if ") && suggested.start_with?("if ") + return "same nil narrowing rewrite" + end + if expected.include?(".compact") && hint_text.include?("remove nil entries") + return "same nil collection filtering rewrite" + end + if expected.include?("|| raise") && hint_text.include?("raise") + return "same nil fallback rewrite" + end + if expected.include?("||") && suggested.include?("||") && hint_text.include?("fallback") + return "same nil fallback operator rewrite" + end + if expected.match?(/\b(?:if|unless)\s+\w+\s*=/) && suggested.match?(/\bif\s+\w+\s*=/) && hint_text.include?("Narrow") + return "same nil binding rewrite" + end + if expected.include?("not_nil!") && (suggested.include?("not_nil!") || suggested.match?(/\bif\s+\w+\s*=/)) + return "same explicit nil assertion rewrite" + end + end + + if wrong.include?("**options") && expected.include?("options[") && suggested.include?("options[") && !suggested.include?("**") + return "same explicit named argument rewrite" + end + + if wrong.match?(/\b(?:while|until)\b/) && expected.match?(/\A(?:while|until)\b/) && suggested.match?(/\A(?:while|until)\b/) + return "same block-form loop rewrite" + end + + if wrong.include?("described_class") && !expected.include?("described_class") && !suggested.include?("described_class") + return "same fully qualified spec subject rewrite" + end + + if wrong.include?(".to_sym") && !expected.include?("to_sym") && !suggested.include?("to_sym") + return "same String/Symbol conversion rewrite" + end + + if wrong.include?("to_i64?") && expected.include?("to_i64") && suggested.include?("to_i64") + return "same integer conversion method correction" + end + + if error.include?("can't cast") && expected.include?("is_a?") && suggested.include?("is_a?") + return "same checked concrete type narrowing rewrite" + end + + if error.include?("expecting token ':'") && error.include?("not '='") && expected.include?(" = ") && suggested.include?(" = ") + return "same inferred constant assignment rewrite" + end + + if error.include?("too many block parameters") && !expected.match?(/\|[^|]+\|/) && !suggested.match?(/\|[^|]+\|/) + return "same zero-parameter block arity correction" + end + + if error.include?("expected block to return") && expected_body = expected[/\{\s*([^{}]+?)\s*\}/, 1] + suggested_body = suggested[/\{\s*([^{}]+?)\s*\}/, 1] + return "same block return value correction" if suggested_body && normalize_code(expected_body) == normalize_code(suggested_body) + end + + if wrong.include?("File.mkdir_p") && expected.include?("Dir.mkdir_p") && suggested.include?("Dir.mkdir_p") + return "same directory API correction" + end + + if wrong.include?("require ") && expected.start_with?("require ") && suggested.start_with?("require ") + expected_path = expected[/require\s+["']([^"']+)["']/, 1] + suggested_path = suggested[/require\s+["']([^"']+)["']/, 1] + if expected_path && suggested_path && relative_path?(expected_path) && relative_path?(suggested_path) + return "same relative require rewrite" + end + end + + if wrong.match?(/can't send closure to C function/) && expected !~ /\b(?:version|app|offset_y|snapshot_path)\b/ && suggested.include?("->") + return "same captureless callback rewrite" + end + + if error.include?("wrong number of block parameters") && expected !~ /\|[^|]+\|/ && suggested !~ /\|[^|]+\|/ + return "same zero-argument block rewrite" + end + + if error.include?("TypeNode#instance_vars") && expected.include?("@type.instance_vars") && suggested.include?("@type.instance_vars") + return "same type-context macro rewrite" + end + + if error.include?("undefined macro variable") && expected.include?("{%") && suggested.include?("{%") + return "same macro variable scope rewrite" + end + + if error.match?(/undefined method '(?:ancestors|superclass)'/) && + (expected.include?("is_a?(") || expected.include?("{{")) && + (suggested.include?("is_a?(") || hint_text.include?("macro")) + return "same runtime or compile-time type hierarchy rewrite" + end + + if error.include?("Base64") && + (expected.include?("Base64.decode(") || expected.include?("Base64.decode_string(")) && + (hint_text.include?("Base64.decode") || hint_text.include?("Base64.decode_string")) + return "same Base64 decoder API correction" + end + + if error.include?("JSON::ParseException.new") && + (expected.include?("JSON::ParseException.new") || expected.include?("ArgumentError.new")) && + (hint_text.include?("parser location arguments") || hint_text.include?("ArgumentError")) + return "same JSON parse error construction correction" + end + + if error.include?("Digest::SHA256") && expected.include?("hexfinal") && suggested.include?("hexfinal") + return "same streaming digest finalization rewrite" + end + + if wrong.include?("String#<<") || (wrong.include?(" << ") && expected.include?(" + ")) + return "same immutable String concatenation rewrite" if expected.include?(" = ") && suggested.include?(" = ") && suggested.include?(" + ") + end + + if wrong.include?("Process.kill") && expected.include?("Process.signal") && suggested.include?("Process.signal") + return "same Process signal API correction" + end + + if wrong.include?("JSON::Any") && expected.include?("as_h") && suggested.include?("as_h") + return "same mutable JSON Hash access rewrite" + end + + if wrong.include?("HTTP::Headers.new") && expected.include?("HTTP::Headers.new") && suggested.include?("HTTP::Headers.new") && suggested.include?(".each") + return "same header copy constructor rewrite" + end + + if wrong.include?("Time.utc") && expected.include?("nanosecond:") && suggested.include?("nanosecond:") + return "same named nanosecond argument rewrite" + end + + if wrong.include?("Slice(UInt8).new") && expected.include?(".dup") && suggested.include?(".dup") + return "same slice copy rewrite" + end + + if wrong.include?("NamedTuple") && expected.include?(".to_h.all?") && suggested.include?(".to_h.all?") + return "same NamedTuple to Hash rewrite" + end + + if wrong.include?("Regex::MatchData#matched") && expected.include?("map(&.[0])") && suggested.include?("map(&.[0])") + return "same regex capture extraction rewrite" + end + + if wrong.include?("String#delete_prefix") && expected.include?("byte_slice") && suggested.include?("byte_slice") + return "same byte slice prefix removal rewrite" + end + + if wrong.include?("Digest::SHA256") && expected.include?("require \"digest/sha256\"") && suggested.include?("require \"digest/sha256\"") + return "same Digest standard library require" + end + + if wrong.include?("FileUtils") && expected.include?("require \"file_utils\"") && suggested.include?("require \"file_utils\"") + return "same FileUtils standard library require" + end + + if error.include?("expecting token") && expected.include?("queries.count {") && suggested.include?("queries.count {") && expected.include?("||") && suggested.include?("||") + return "same explicit block predicate rewrite" + end + + if error.include?("can't return from captured block") && expected.include?("next") && suggested.include?("next") + return "same block-local next rewrite" + end + + if error.include?("can't declare def dynamically") && expected.include?("private def") && suggested.include?("private def") + return "same file-scope spec helper rewrite" + end + + if error.include?("can't declare constant dynamically") && (expected.include?("private ") || expected.include?("module ")) && hint_text.include?("file or module scope") + return "same file-scope spec constant rewrite" + end + + if error.include?("cannot use") && (expected.include?("macro_name") || expected.include?("protected_environments")) && + (suggested.include?("macro_name") || suggested.include?("protected_environments")) + return "same reserved identifier rename" + end + + if error.include?("can't infer the type of instance variable") && expected.include?("@") && expected.include?(":") && suggested.include?("@") && suggested.include?(":") + return "same instance variable type declaration" + end + + if error.include?("doesn't explicitly initialize instance variable") && expected.include?("@") && suggested.include?("@") + return "same instance variable initialization correction" + end + + nil +end + +def relative_path?(path) + path.start_with?("./") || path.start_with?("../") +end + +def evaluate_case(item, binary_path, project_root) + error_text = [item[:error], item[:compiler_snippet]].reject(&:empty?).join("\n") + stdout, stderr, status = Open3.capture3( + binary_path, + "hint", + "--root", + project_root, + stdin_data: error_text, + chdir: REPO_ROOT, + ) + unless status.success? + raise "hint command failed for #{item[:case_id]} (#{status.exitstatus}): #{stderr.strip}" + end + + hints = parse_hint_output(stdout) + matched_hint = nil + matched_rule = nil + hints.each do |hint| + match_rule = right_form_match_rule(item, hint) + next unless match_rule + + matched_hint = hint + matched_rule = match_rule + break + end + + { + :case_id => item[:case_id], + :category => item[:category], + :split => item[:split], + :right => item[:right], + :error => item[:error], + :compiler_snippet => item[:compiler_snippet], + :matched => !matched_hint.nil?, + :match_rule => matched_rule, + :matched_hint => matched_hint, + :hints => hints, + } +end + +def category_summary(list_of_results, category, split) + rows = list_of_results.select { |item| item[:category] == category && item[:split] == split } + hit_count = rows.count { |item| item[:matched] } + { + :hits => hit_count, + :total => rows.size, + :rate => rows.empty? ? 0.0 : (100.0 * hit_count / rows.size), + } +end + +def evaluate_card_family_support(card, authoring_cases) + card.fetch("error_hints").map do |hint| + pattern = Regexp.new(hint.fetch("pattern").gsub("(?|", "(?:")) + matched_cases = authoring_cases.count do |item| + compiler_text = [item[:error], item[:compiler_snippet]].reject(&:empty?).join("\n") + !!pattern.match(compiler_text) + end + { + :pattern => hint.fetch("pattern"), + :authoring_cases => matched_cases, + } + rescue RegexpError => error + { + :pattern => hint.fetch("pattern"), + :authoring_cases => 0, + :regex_error => error.message, + } + end +end + +def short_cell(value, limit = 220) + text = value.to_s.gsub(/\s+/, " ").strip + text = "(none)" if text.empty? + text = "#{text[0, limit]}…" if text.size > limit + text.gsub("|") { "\\|" }.gsub("`") { "\\`" } +end + +def markdown_report(results, options, all_cases, label_files) + case_sha = Digest::SHA256.file(options[:cases_path]).hexdigest + label_sha = Digest::SHA256.new + label_files.each do |path| + label_sha.update(File.basename(path)) + label_sha.update(File.binread(path)) + end + commit, = Open3.capture2("git", "rev-parse", "HEAD", chdir: REPO_ROOT) + + report = [] + report << "# Crystal language card hint evaluation" + report << "" + report << "- Measured: #{Time.now.utc.strftime("%Y-%m-%d %H:%M:%S UTC")}." + report << "- Card commit: `#{commit.strip}`." + report << "- Heldout measurement label: `#{options[:heldout_label]}` (the initial post-freeze heldout run)." + report << "- Split assignment: even final digit of the numeric `case_id` suffix is authoring; odd is heldout." + report << "- Cases: #{all_cases.size} trainable joined records; #{authoring_cases.size} authoring and #{all_cases.size - authoring_cases.size} heldout." + report << "- `cases_labeled.jsonl` SHA-256: `#{case_sha}`." + report << "- Sorted label JSONL set SHA-256: `#{label_sha.hexdigest}`." + report << "- Matching rules: normalized equality, containment, then the family-specific same-rewrite checks implemented in `scripts/crystal_card_eval.rb`." + report << "" + report << "## Hit rates" + report << "" + report << "| Category | Split | Hits | Cases | Rate |" + report << "|---|---:|---:|---:|---:|" + ["authoring", "heldout"].each do |split| + next unless results.any? { |item| item[:split] == split } + + CATEGORY_TOTALS.keys.each do |category| + summary = category_summary(results, category, split) + report << "| #{category} | #{split} | #{summary[:hits]} | #{summary[:total]} | #{format("%.1f%%", summary[:rate])} |" + end + rows = results.select { |item| item[:split] == split } + hit_count = rows.count { |item| item[:matched] } + rate = rows.empty? ? 0.0 : 100.0 * hit_count / rows.size + report << "| **Overall** | #{split} | **#{hit_count}** | **#{rows.size}** | **#{format("%.1f%%", rate)}** |" + end + + report << "" + report << "## Misses" + report << "" + report << "| Split | Category | Case | Expected right form | Hints emitted |" + report << "|---|---|---|---|---|" + results.reject { |item| item[:matched] }.each do |item| + suggestions = item[:hints].map { |hint| hint[:example] }.reject(&:empty?).join("; ") + if suggestions.empty? + suggestions = "(no matching hint)" + end + report << "| #{item[:split]} | #{item[:category]} | `#{item[:case_id]}` | #{short_cell(item[:right])} | #{short_cell(suggestions)} |" + end + report << "" + report.join("\n") + "\n" +end + +def print_summary(results, options) + puts "Split: #{options[:split]}" + puts "Heldout label: #{options[:heldout_label]}" unless options[:heldout_label].to_s.empty? + selected_splits = results.map { |item| item[:split] }.uniq + selected_splits.each do |split| + CATEGORY_TOTALS.keys.each do |category| + summary = category_summary(results, category, split) + puts "#{split} #{category}: #{summary[:hits]}/#{summary[:total]} (#{format("%.1f%%", summary[:rate])})" + end + rows = results.select { |item| item[:split] == split } + hit_count = rows.count { |item| item[:matched] } + rate = rows.empty? ? 0.0 : 100.0 * hit_count / rows.size + puts "#{split} overall: #{hit_count}/#{rows.size} (#{format("%.1f%%", rate)})" + end + + if options[:show_misses] + misses = results.reject { |item| item[:matched] } + puts "Misses: #{misses.size}" + misses.each do |item| + suggestions = item[:hints].map { |hint| hint[:example] }.reject(&:empty?).join(" / ") + suggestions = "(no matching hint)" if suggestions.empty? + puts "MISS #{item[:case_id]} #{item[:category]} expected=#{short_cell(item[:right], 180)} suggestions=#{short_cell(suggestions, 180)}" + end + end + + if options[:show_misses] && results.any? { |item| item[:split] == "authoring" } + card = YAML.load_file(CARD_PATH) + authoring_cases = results.select { |item| item[:split] == "authoring" } + puts "Card pattern authoring matches:" + evaluate_card_family_support(card, authoring_cases).each_with_index do |family, index| + puts "#{index + 1}. #{family[:authoring_cases]} cases: #{family[:pattern]}" + end + end +end + +def run + options = parse_options + binary_path = File.expand_path(options[:binary_path]) + raise "amber-lsp executable is not available: #{binary_path}" unless File.executable?(binary_path) + raise "Crystal card is missing: #{CARD_PATH}" unless File.file?(CARD_PATH) + + case_records = read_case_records(options[:cases_path]) + all_cases = trainable_labeled_cases(options[:labels_path], case_records) + selected_splits = case options[:split] + when "authoring" + ["authoring"] + when "heldout" + ["heldout"] + else + ["authoring", "heldout"] + end + selected_cases = all_cases.select { |item| selected_splits.include?(item[:split]) } + + started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC) + results = [] + Dir.mktmpdir("amber-lsp-plain-project") do |project_root| + selected_cases.each do |item| + results << evaluate_case(item, binary_path, project_root) + end + end + elapsed_seconds = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at + + print_summary(results, options) + puts "Evaluated #{results.size} cases in #{format("%.2f", elapsed_seconds)} seconds." + + if options[:report_path] + label_files = Dir.glob(File.join(options[:labels_path], "*.jsonl")).sort + report = markdown_report(results, options, all_cases, label_files) + FileUtils.mkdir_p(File.dirname(File.expand_path(options[:report_path]))) + File.write(options[:report_path], report) + puts "Report: #{File.expand_path(options[:report_path])}" + end + + true +rescue OptionParser::ParseError, Errno::ENOENT, KeyError, JSON::ParserError, RegexpError, RuntimeError => error + warn "crystal card eval failed: #{error.message}" + false +end + +exit(run ? 0 : 1) From d3980f816ff22f2ecbeb027f2241ebaa00fae895 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 05:31:56 -0400 Subject: [PATCH 58/69] test(agent): assert readiness cache hit by invocation count --- spec/commands/agent_hook_spec.cr | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/spec/commands/agent_hook_spec.cr b/spec/commands/agent_hook_spec.cr index 5e6fe2b..4094910 100644 --- a/spec/commands/agent_hook_spec.cr +++ b/spec/commands/agent_hook_spec.cr @@ -154,7 +154,7 @@ describe "amber-agent-hook readiness and preflight" do end end - it "checks a verified binary and project coverage once, then serves the cached result under 300 ms" do + it "checks a verified binary and project coverage once, then serves the cached result without another binary call" do SpecHelper.within_temp_directory do |project_root| fixture = AgentHookSpecHelper.create_ready_project(project_root) @@ -163,14 +163,11 @@ describe "amber-agent-hook readiness and preflight" do first_check[1].should eq("Amber agent setup is ready.\n") File.read(fixture.lsp_log).should contain("--version") File.read(fixture.lsp_log).should contain("--check src/app.cr") + File.read(fixture.lsp_log).lines.size.should eq(2) - previous_calls = File.read(fixture.lsp_log) - start_time = Time.instant cached_check = AgentHookSpecHelper.run_hook(fixture, "check") - elapsed_milliseconds = (Time.instant - start_time).total_milliseconds cached_check[0].should eq(0) - File.read(fixture.lsp_log).should eq(previous_calls) - elapsed_milliseconds.should be < 300.0 + File.read(fixture.lsp_log).lines.size.should eq(2) end end From 647da53f785723bc90d466154fdaa1b4d86081ba Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 05:40:15 -0400 Subject: [PATCH 59/69] fix(release): support macOS Bash platform flags --- .github/workflows/release.yml | 2 +- scripts/build_release.sh | 18 +++++++++++++----- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d1a0537..6cbfb48 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -67,7 +67,7 @@ jobs: CRYSTAL: crystal-alpha - name: Install dependencies - run: minecart install --production --skip-ai-docs + run: minecart install --frozen --production --skip-ai-docs - name: Build binaries (Linux) if: startsWith(matrix.target, 'linux-') diff --git a/scripts/build_release.sh b/scripts/build_release.sh index dda0eb9..6ce0727 100755 --- a/scripts/build_release.sh +++ b/scripts/build_release.sh @@ -22,7 +22,7 @@ mkdir -p "$BUILD_DIR" "$PACKAGE_ROOT/share/amber_cli/api" OS=$(uname -s | tr '[:upper:]' '[:lower:]') ARCH=$(uname -m) -PLATFORM_FLAGS=() +PLATFORM_FLAG="" case "$OS" in darwin) @@ -42,7 +42,7 @@ case "$OS" in exit 1 ;; esac - PLATFORM_FLAGS=(--static) + PLATFORM_FLAG="--static" CHECKSUM_CMD=(sha256sum) ;; *) @@ -55,17 +55,25 @@ echo "Building for target: $TARGET" echo "Installing dependencies..." if command -v minecart >/dev/null 2>&1; then - minecart install --production --skip-ai-docs + minecart install --frozen --production --skip-ai-docs else command -v shards-alpha >/dev/null 2>&1 || { echo "minecart or shards-alpha is required" >&2; exit 1; } shards-alpha install --production --frozen fi echo "Compiling amber..." -crystal-alpha build src/amber_cli.cr -o "$BUILD_DIR/amber" --release "${PLATFORM_FLAGS[@]}" +if [ -n "$PLATFORM_FLAG" ]; then + crystal-alpha build src/amber_cli.cr -o "$BUILD_DIR/amber" --release "$PLATFORM_FLAG" +else + crystal-alpha build src/amber_cli.cr -o "$BUILD_DIR/amber" --release +fi echo "Compiling amber-lsp..." -crystal-alpha build src/amber_lsp.cr -o "$BUILD_DIR/amber-lsp" --release "${PLATFORM_FLAGS[@]}" +if [ -n "$PLATFORM_FLAG" ]; then + crystal-alpha build src/amber_lsp.cr -o "$BUILD_DIR/amber-lsp" --release "$PLATFORM_FLAG" +else + crystal-alpha build src/amber_lsp.cr -o "$BUILD_DIR/amber-lsp" --release +fi "$BUILD_DIR/amber" --version "$BUILD_DIR/amber-lsp" --version From dc97420433e9b8bab20c7f92377264156b22740f Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 07:19:32 -0400 Subject: [PATCH 60/69] fix(agent): allow plain Crystal readiness --- spec/commands/agent_hook_spec.cr | 42 ++++++++++++++++- spec/commands/doctor_spec.cr | 45 ++++++++++++++++++- spec/commands/setup_agent_spec.cr | 32 +++++++++++-- src/amber_cli/agent/agent_setup_guidance.cr | 24 ++++++++-- .../check_amber_agent_setup_command.cr | 5 ++- src/amber_cli/commands/setup_agent_command.cr | 37 +++++++++++---- .../templates/agent/amber-agent-hook | 31 ++++++++++--- 7 files changed, 190 insertions(+), 26 deletions(-) diff --git a/spec/commands/agent_hook_spec.cr b/spec/commands/agent_hook_spec.cr index 4094910..b9d14e9 100644 --- a/spec/commands/agent_hook_spec.cr +++ b/spec/commands/agent_hook_spec.cr @@ -55,7 +55,11 @@ module AgentHookSpecHelper codex_settings = AmberCLI::Agent::MergeAgentHooksIntoSettings.new("", false).perform File.write(".codex/hooks.json", codex_settings) - script = AmberCLI::Commands::SetupAgentCommand::AGENT_HOOK_SCRIPT.gsub("__AMBER_MAIN__", "src/app.cr") + script = AmberCLI::Commands::SetupAgentCommand::AGENT_HOOK_SCRIPT + .gsub("__AMBER_MAIN__", "src/app.cr") + .gsub("__AMBER_LOOKUP_INSTRUCTION__", AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION) + .gsub("__PLAIN_CRYSTAL_LOOKUP_INSTRUCTION__", AmberCLI::Agent::AgentSetupGuidance::PLAIN_CRYSTAL_REQUIRED_LOOKUP_INSTRUCTION) + .gsub("__PLAIN_CRYSTAL_READINESS_ITEM__", AmberCLI::Agent::AgentSetupGuidance::PLAIN_CRYSTAL_READINESS_ITEM) hook_path = File.join(project_root, ".amber/amber-agent-hook") File.write(hook_path, script) File.chmod(hook_path, 0o755) @@ -154,6 +158,42 @@ describe "amber-agent-hook readiness and preflight" do end end + it "accepts declined coverage for a plain Crystal project after verifying the LSP" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project( + project_root, + coverage: "amber-lsp: declined project is not an Amber V2 stack project", + coverage_exit: "2", + ) + + check = AgentHookSpecHelper.run_hook(fixture, "check") + session = AgentHookSpecHelper.run_hook(fixture, "session") + + check[0].should eq(0) + check[1].should eq("plain Crystal project: Amber rules do not apply; lookup and Crystal hints are active\n") + session[0].should eq(0) + session[1].should contain("plain Crystal project: Amber rules do not apply; lookup and Crystal hints are active") + session[1].should contain("amber-lsp version: 1.0.0") + session[1].should contain("Before writing code that calls Crystal standard library methods or methods from this project's own shards, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints.") + session[1].should_not contain("Before writing code that calls Grant, Amber, asset_pipeline") + end + end + + it "still fails when amber-lsp reports failed coverage" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project( + project_root, + coverage: "amber-lsp: failed invalid project configuration", + coverage_exit: "1", + ) + + check = AgentHookSpecHelper.run_hook(fixture, "check") + + check[0].should eq(2) + check[1].should contain("amber-lsp does not cover this project main entry file") + end + end + it "checks a verified binary and project coverage once, then serves the cached result without another binary call" do SpecHelper.within_temp_directory do |project_root| fixture = AgentHookSpecHelper.create_ready_project(project_root) diff --git a/spec/commands/doctor_spec.cr b/spec/commands/doctor_spec.cr index a27fbbc..692d702 100644 --- a/spec/commands/doctor_spec.cr +++ b/spec/commands/doctor_spec.cr @@ -1,11 +1,14 @@ require "../amber_cli_spec" +require "digest/sha256" require "../../src/amber_cli/commands/check_amber_agent_setup_command" +require "../../src/amber_cli/commands/setup_agent_command" require "../../src/amber_cli/agent/install_claude_amber_lsp_plugin" require "../../src/amber_cli/agent/agent_setup_manifest" require "../../src/amber_cli/agent/agent_setup_guidance" module DoctorCommandSpecHelper def self.create_agent_settings(project_root : String) : String + File.write(File.join(project_root, "shard.yml"), "name: amber_cli\nversion: 0.1.0\n") Dir.mkdir_p(File.join(project_root, ".claude")) Dir.mkdir_p(File.join(project_root, ".codex")) Dir.mkdir_p(File.join(project_root, ".amber")) @@ -105,7 +108,7 @@ describe "amber doctor" do Dir.mkdir_p(File.join(project_root, ".amber")) old_guidance = "Before using a library API you are not sure of, run `amber-lsp lookup 'Type.method'`." File.write(File.join(project_root, ".amber/agent_setup.json"), <<-JSON) - {"amber_cli_version":"2.0.7","minimum_amber_lsp_version":"1.0.0","generated_hook_version":"4"} + {"amber_cli_version":"2.0.7","minimum_amber_lsp_version":"1.0.0","generated_hook_version":"5"} JSON File.write(File.join(project_root, ".amber/amber-agent-hook"), "#!/bin/sh\n#{old_guidance}\n") File.write(File.join(project_root, "CLAUDE.md"), "\n#{old_guidance}\n\n") @@ -118,6 +121,46 @@ describe "amber doctor" do end end + it "passes the plain Crystal readiness item when the verified LSP declines Amber coverage" do + SpecHelper.within_temp_directory do |project_root| + Dir.mkdir_p(File.join(project_root, "src")) + File.write(File.join(project_root, "shard.yml"), "name: plain_app\nversion: 0.1.0\n") + File.write(File.join(project_root, "src/plain_app.cr"), "puts :ok\n") + AmberCLI::Commands::SetupAgentCommand.new("setup:agent").execute + + home_directory = File.join(project_root, "home") + tools_directory = File.join(project_root, "tools") + Dir.mkdir_p(home_directory) + Dir.mkdir_p(tools_directory) + lsp_path = File.join(tools_directory, "amber-lsp") + DoctorCommandSpecHelper.write_executable(lsp_path, <<-SH) + #!/bin/sh + case "$1" in + --version) printf '%s\\n' 'amber-lsp 1.0.0'; exit 0 ;; + --check) printf '%s\\n' 'amber-lsp: declined project is not an Amber V2 stack project'; exit 2 ;; + lookup) printf '%s\\n' 'Amber API index is fresh'; exit 0 ;; + esac + exit 2 + SH + checksum = Digest::SHA256.hexdigest(File.read(lsp_path)) + File.write("#{lsp_path}.sha256", "#{checksum} #{lsp_path}\n") + + previous_path = ENV["PATH"]? + previous_lsp = ENV["AMBER_LSP_BIN"]? + ENV["PATH"] = "#{tools_directory}:/usr/bin:/bin" + ENV["AMBER_LSP_BIN"] = lsp_path + begin + report = AmberCLI::Commands::CheckAmberAgentSetupCommand.new("doctor", project_root, home_directory).perform + + report.to_s.should contain("[PASS] plain Crystal project: Amber rules do not apply; lookup and Crystal hints are active") + report.to_s.should_not contain("amber-lsp does not cover this project main entry file") + ensure + DoctorCommandSpecHelper.restore_environment("PATH", previous_path) + DoctorCommandSpecHelper.restore_environment("AMBER_LSP_BIN", previous_lsp) + end + end + end + it "reports a ready project with exit status 0 when hooks, trust, plugins, and the API index pass" do SpecHelper.within_temp_directory do |project_root| home_directory = File.join(project_root, "home") diff --git a/spec/commands/setup_agent_spec.cr b/spec/commands/setup_agent_spec.cr index 866343b..e16eb44 100644 --- a/spec/commands/setup_agent_spec.cr +++ b/spec/commands/setup_agent_spec.cr @@ -97,7 +97,7 @@ describe "amber setup:agent" do SpecHelper.within_temp_directory do |project| Dir.mkdir_p("src") File.write("src/custom_entry.cr", "puts :ok\n") - File.write("shard.yml", "name: my_app\ntargets:\n app_entry:\n main: src/custom_entry.cr\n") + File.write("shard.yml", "name: amber_cli\ntargets:\n app_entry:\n main: src/custom_entry.cr\n") File.write("CLAUDE.md", "# Existing Claude instructions\n") File.write("AGENTS.md", "# Existing Codex instructions\n") Dir.mkdir_p(".claude") @@ -130,7 +130,7 @@ describe "amber setup:agent" do setup_manifest = AmberCLI::Agent::AgentSetupManifest.from_json(File.read(".amber/agent_setup.json")) setup_manifest.amber_cli_version.should eq(AmberCli::VERSION) setup_manifest.minimum_amber_lsp_version.should eq("1.0.0") - setup_manifest.generated_hook_version.should eq("5") + setup_manifest.generated_hook_version.should eq("6") File.file?(".amber/claude-marketplace/amber-lsp/.lsp.json").should be_true File.file?(".lsp.json").should be_false File.info(".amber/amber-agent-hook").permissions.to_i.&(0o111).should_not eq(0) @@ -303,11 +303,35 @@ describe "amber setup:agent" do end end + it "sets up plain Crystal without adding Amber configuration or dependencies" do + SpecHelper.within_temp_directory do + Dir.mkdir_p("src") + File.write("shard.yml", "name: plain_app\nversion: 0.1.0\n") + File.write("src/plain_app.cr", "puts :ok\n") + + command = RecordSetupAgentMessages.new("setup:agent") + command.execute + + Dir.exists?("config").should be_false + File.exists?(".amber-lsp.yml").should be_false + File.read("shard.yml").should_not contain("dependencies:") + File.file?(".amber/amber-agent-hook").should be_true + File.file?(".amber/agent_setup.json").should be_true + File.file?(".amber/claude-marketplace/.claude-plugin/marketplace.json").should be_true + File.file?(".amber/claude-marketplace/amber-lsp/.claude-plugin/plugin.json").should be_true + File.file?(".claude/settings.json").should be_true + File.file?(".codex/hooks.json").should be_true + File.read("CLAUDE.md").should contain("Before writing code that calls Crystal standard library methods or methods from this project's own shards") + File.read("AGENTS.md").should contain("Before writing code that calls Crystal standard library methods or methods from this project's own shards") + File.read("CLAUDE.md").should_not contain("Before writing code that calls Grant, Amber, asset_pipeline") + end + end + it "corrects older marked agent instructions on rerun" do SpecHelper.within_temp_directory do Dir.mkdir_p("src") - File.write("shard.yml", "name: my_app\n") - File.write("src/my_app.cr", "puts :ok\n") + File.write("shard.yml", "name: amber_cli\n") + File.write("src/amber_cli.cr", "puts :ok\n") older_instructions = "# Keep this\n\n\n## Agent loop\n\nUse `crystal spec --affected` when available.\nBefore using a library API you are not sure of, run\n`amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover.\nThe SessionStart hook reports whether setup is complete.\n\n" File.write("CLAUDE.md", older_instructions) File.write("AGENTS.md", older_instructions) diff --git a/src/amber_cli/agent/agent_setup_guidance.cr b/src/amber_cli/agent/agent_setup_guidance.cr index bb3c63b..59fd0e4 100644 --- a/src/amber_cli/agent/agent_setup_guidance.cr +++ b/src/amber_cli/agent/agent_setup_guidance.cr @@ -1,7 +1,23 @@ +require "../../amber_lsp/project_context" + module AmberCLI::Agent::AgentSetupGuidance - GENERATED_HOOK_VERSION = "5" - DOCUMENT_START = "" - DOCUMENT_END = "" + GENERATED_HOOK_VERSION = "6" + DOCUMENT_START = "" + DOCUMENT_END = "" + PLAIN_CRYSTAL_READINESS_ITEM = "plain Crystal project: Amber rules do not apply; lookup and Crystal hints are active" + + REQUIRED_LOOKUP_INSTRUCTION = "Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints." + PLAIN_CRYSTAL_REQUIRED_LOOKUP_INSTRUCTION = "Before writing code that calls Crystal standard library methods or methods from this project's own shards, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints." + + def self.uses_amber_v2_stack?(project_root : String) : Bool + AmberLSP::ProjectContext.detect(project_root).stack_project? + end - REQUIRED_LOOKUP_INSTRUCTION = "Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints." + def self.required_lookup_instruction_for(project_root : String) : String + if uses_amber_v2_stack?(project_root) + REQUIRED_LOOKUP_INSTRUCTION + else + PLAIN_CRYSTAL_REQUIRED_LOOKUP_INSTRUCTION + end + end end diff --git a/src/amber_cli/commands/check_amber_agent_setup_command.cr b/src/amber_cli/commands/check_amber_agent_setup_command.cr index f1922e3..5dafd80 100644 --- a/src/amber_cli/commands/check_amber_agent_setup_command.cr +++ b/src/amber_cli/commands/check_amber_agent_setup_command.cr @@ -180,7 +180,7 @@ module AmberCLI::Commands private def check_generated_agent_instructions : DoctorCheck setup_manifest_path = File.join(@project_root, ".amber/agent_setup.json") hook_path = File.join(@project_root, ".amber/amber-agent-hook") - required_instruction = AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION + required_instruction = AmberCLI::Agent::AgentSetupGuidance.required_lookup_instruction_for(@project_root) has_current_manifest = false if File.file?(setup_manifest_path) setup_manifest = AmberCLI::Agent::AgentSetupManifest.from_json(File.read(setup_manifest_path)) @@ -225,6 +225,9 @@ module AmberCLI::Commands exit_code, output, errors = run_process(hook_path, ["check"], @project_root) if exit_code == 0 + if output.lines.any? { |line| line.strip == AmberCLI::Agent::AgentSetupGuidance::PLAIN_CRYSTAL_READINESS_ITEM } + return [DoctorCheck.new("PASS", AmberCLI::Agent::AgentSetupGuidance::PLAIN_CRYSTAL_READINESS_ITEM)] + end [DoctorCheck.new("PASS", "Amber agent readiness check passed.")] else list_of_readiness_failures = (output + errors).lines.map(&.strip).reject(&.empty?) diff --git a/src/amber_cli/commands/setup_agent_command.cr b/src/amber_cli/commands/setup_agent_command.cr index bf7cfd3..0832433 100644 --- a/src/amber_cli/commands/setup_agent_command.cr +++ b/src/amber_cli/commands/setup_agent_command.cr @@ -18,7 +18,7 @@ module AmberCLI::Commands DOCUMENT_END = AmberCLI::Agent::AgentSetupGuidance::DOCUMENT_END def help_description : String - "Set up the Claude Code and Codex agent loop in an Amber V2 project" + "Set up the Claude Code and Codex agent loop for a Crystal project" end def setup_command_options @@ -26,7 +26,16 @@ module AmberCLI::Commands def execute main_file = find_project_main_file - SetupLSPCommand.new("setup:lsp").execute + uses_amber_stack = AmberCLI::Agent::AgentSetupGuidance.uses_amber_v2_stack?(Dir.current) + required_lookup_instruction = AmberCLI::Agent::AgentSetupGuidance.required_lookup_instruction_for(Dir.current) + if uses_amber_stack + SetupLSPCommand.new("setup:lsp").execute + else + info "Setting up the Amber LSP Claude plugin..." + list_of_updated_paths = AmberCLI::Agent::InstallClaudeAmberLSPPlugin.new.perform + list_of_updated_paths.each { |path| info "Updated: #{path}" } + success "Amber LSP plugin setup complete." + end [{".claude/settings.json", true}, {".codex/hooks.json", false}].each do |path, is_claude_settings| write_merged_hooks(path, is_claude_settings) end @@ -34,14 +43,14 @@ module AmberCLI::Commands write_setup_manifest warn_about_ignored_agent_hook_settings ["CLAUDE.md", "AGENTS.md"].each do |path| - append_agent_loop_instructions(path) + append_agent_loop_instructions(path, required_lookup_instruction) end success "Amber agent loop installed." end private def find_project_main_file : String unless File.file?("shard.yml") - raise "Run amber setup:agent from an Amber project with shard.yml" + raise "Run amber setup:agent from a Crystal project with shard.yml" end manifest = YAML.parse(File.read("shard.yml")) @@ -80,7 +89,11 @@ module AmberCLI::Commands private def write_hook_script(main_file : String) : Nil path = ".amber/amber-agent-hook" - content = AGENT_HOOK_SCRIPT.gsub("__AMBER_MAIN__", main_file) + content = AGENT_HOOK_SCRIPT + .gsub("__AMBER_MAIN__", main_file) + .gsub("__AMBER_LOOKUP_INSTRUCTION__", AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION) + .gsub("__PLAIN_CRYSTAL_LOOKUP_INSTRUCTION__", AmberCLI::Agent::AgentSetupGuidance::PLAIN_CRYSTAL_REQUIRED_LOOKUP_INSTRUCTION) + .gsub("__PLAIN_CRYSTAL_READINESS_ITEM__", AmberCLI::Agent::AgentSetupGuidance::PLAIN_CRYSTAL_READINESS_ITEM) Dir.mkdir_p(".amber") if !File.file?(path) || File.read(path) != content File.write(path, content) @@ -99,7 +112,7 @@ module AmberCLI::Commands File.write(".amber/agent_setup.json", manifest.to_pretty_json + "\n") end - private def append_agent_loop_instructions(path : String) : Nil + private def append_agent_loop_instructions(path : String, required_lookup_instruction : String) : Nil content = File.file?(path) ? File.read(path) : "" if content.includes?(DOCUMENT_START) && content.includes?(DOCUMENT_END) marker_start = content.index(DOCUMENT_START) @@ -115,8 +128,14 @@ module AmberCLI::Commands "Before using a library API you are not sure of, run `amber-lsp lookup 'Type.method'` or use the LSP tool's workspaceSymbol/hover.", "", ) - unless updated_section.includes?(AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION) - updated_section += "\n#{AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION}" + obsolete_lookup_instruction = if required_lookup_instruction == AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION + AmberCLI::Agent::AgentSetupGuidance::PLAIN_CRYSTAL_REQUIRED_LOOKUP_INSTRUCTION + else + AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION + end + updated_section = updated_section.gsub(obsolete_lookup_instruction, "") + unless updated_section.includes?(required_lookup_instruction) + updated_section += "\n#{required_lookup_instruction}" end unless updated_section.includes?("stop and tell the user to run `amber setup:agent`") updated_section += "\nThe SessionStart hook reports whether setup is complete. When setup is missing, stop and tell the user to run `amber setup:agent` before editing Crystal files." @@ -138,7 +157,7 @@ module AmberCLI::Commands Use `crystal-alpha spec --affected` when available. Format Crystal files with `crystal-alpha tool format`. The installed hooks hold the watcher during edits, check each changed file, and build when the agent stops. - #{AmberCLI::Agent::AgentSetupGuidance::REQUIRED_LOOKUP_INSTRUCTION} + #{required_lookup_instruction} The SessionStart hook reports whether setup is complete. When setup is missing, stop and tell the user to run `amber setup:agent` before editing Crystal files. diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook index b1483ea..3f75f2e 100644 --- a/src/amber_cli/templates/agent/amber-agent-hook +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -4,7 +4,7 @@ set -u project_root=$(CDPATH='' cd "$(dirname "$0")/.." && pwd -P) || exit 2 cd "$project_root" || exit 2 -HOOK_VERSION=5 +HOOK_VERSION=6 DEFAULT_MINIMUM_AMBER_LSP_VERSION=1.0.0 MAIN_FILE='__AMBER_MAIN__' @@ -354,13 +354,14 @@ setup_cache_key() { fi done cache_project=$(printf '%s' "$project_root" | cksum) - cache_identity=$(printf '%s\n%s\n%s\n%s' "$lsp_path" "$lsp_mtime" "$setup_signature" "$dependent_mtimes" | cksum) + cache_identity=$(printf '%s\n%s\n%s\n%s\n%s' "$HOOK_VERSION" "$lsp_path" "$lsp_mtime" "$setup_signature" "$dependent_mtimes" | cksum) CHECK_CACHE_KEY="$cache_project:$cache_identity" } compute_setup_check() { CHECK_FAILURES='' CHECK_VERSION='unknown' + CHECK_PROJECT_KIND='amber-stack' lsp=$(resolve_lsp 2>/dev/null || :) setup_cache_key "$lsp" cache_dir=${TMPDIR:-/tmp} @@ -368,10 +369,12 @@ compute_setup_check() { if [ -f "$cache_file" ]; then cached_key='' cached_version='unknown' + cached_project_kind='amber-stack' cached_failures='' { IFS= read -r cached_key || : IFS= read -r cached_version || : + IFS= read -r cached_project_kind || cached_project_kind='amber-stack' while IFS= read -r cached_failure; do if [ -n "$cached_failures" ]; then cached_failures="${cached_failures} @@ -383,6 +386,7 @@ ${cached_failure}" } < "$cache_file" if [ "$cached_key" = "$CHECK_CACHE_KEY" ]; then CHECK_VERSION=$cached_version + CHECK_PROJECT_KIND=$cached_project_kind CHECK_FAILURES=$cached_failures return fi @@ -449,6 +453,9 @@ ${cached_failure}" coverage_output=$("$lsp" --check "$MAIN_FILE" 2>&1) coverage_status=$? case "$coverage_output" in + 'amber-lsp: declined project is not an Amber V2 stack project') + CHECK_PROJECT_KIND='plain-crystal' + ;; *declined*|*failed*) add_check_failure "amber-lsp does not cover this project main entry file; update the Amber LSP rules and run \`amber setup:agent\` again." ;; @@ -482,6 +489,7 @@ ${cached_failure}" if { printf '%s\n' "$CHECK_CACHE_KEY" printf '%s\n' "$CHECK_VERSION" + printf '%s\n' "$CHECK_PROJECT_KIND" [ -z "$CHECK_FAILURES" ] || printf '%s\n' "$CHECK_FAILURES" } > "$cache_temp" 2>/dev/null && mv "$cache_temp" "$cache_file" 2>/dev/null; then : @@ -626,7 +634,11 @@ changed_cr_files() { case "${1:-}" in check) if run_setup_check; then - printf '%s\n' 'Amber agent setup is ready.' + if [ "$CHECK_PROJECT_KIND" = 'plain-crystal' ]; then + printf '%s\n' '__PLAIN_CRYSTAL_READINESS_ITEM__' + else + printf '%s\n' 'Amber agent setup is ready.' + fi exit 0 fi print_setup_failures @@ -634,11 +646,18 @@ case "${1:-}" in ;; session) if run_setup_check; then - printf '%s\n' 'Amber agent setup is OK.' printf 'amber-lsp version: %s\n' "$CHECK_VERSION" - cat <<'AMBER_LOOKUP_GUIDANCE' -Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints. + if [ "$CHECK_PROJECT_KIND" = 'plain-crystal' ]; then + printf '%s\n' '__PLAIN_CRYSTAL_READINESS_ITEM__' + cat <<'PLAIN_CRYSTAL_LOOKUP_GUIDANCE' +__PLAIN_CRYSTAL_LOOKUP_INSTRUCTION__ +PLAIN_CRYSTAL_LOOKUP_GUIDANCE + else + printf '%s\n' 'Amber agent setup is OK.' + cat <<'AMBER_LOOKUP_GUIDANCE' +__AMBER_LOOKUP_INSTRUCTION__ AMBER_LOOKUP_GUIDANCE + fi else print_setup_refusal fi From 2daf3a41831649b374162e5a975d74f2540af2a2 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 09:53:34 -0400 Subject: [PATCH 61/69] fix(agent): keep post-edit hook quiet in plain Crystal projects amber-lsp --check declines projects that are not Amber V2 stacks. The post-edit hook passed that decline through as exit 2, so agents saw a failure after every .cr edit in a plain Crystal app or shard. Formatting still runs; Amber findings still fail in covered projects. --- spec/commands/agent_hook_spec.cr | 38 +++++++++++++++++++ .../templates/agent/amber-agent-hook | 2 + 2 files changed, 40 insertions(+) diff --git a/spec/commands/agent_hook_spec.cr b/spec/commands/agent_hook_spec.cr index b9d14e9..edecf04 100644 --- a/spec/commands/agent_hook_spec.cr +++ b/spec/commands/agent_hook_spec.cr @@ -179,6 +179,44 @@ describe "amber-agent-hook readiness and preflight" do end end + it "formats a plain Crystal project's edited file without reporting the declined rule check" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project( + project_root, + coverage: "amber-lsp: declined project is not an Amber V2 stack project", + coverage_exit: "2", + ) + compiler_path = File.join(project_root, "fake-tools", "crystal-alpha") + File.write(compiler_path, "#!/bin/sh\nexit 0\n") + File.chmod(compiler_path, 0o755) + payload = {"tool_name" => "Edit", "tool_input" => {"file_path" => File.join(project_root, "src/app.cr")}}.to_json + + post = AgentHookSpecHelper.run_hook(fixture, "post", payload) + + post[0].should eq(0) + post[2].should eq("") + end + end + + it "reports a covered project's post-edit findings" do + SpecHelper.within_temp_directory do |project_root| + fixture = AgentHookSpecHelper.create_ready_project( + project_root, + coverage: "src/app.cr:1:1 error: unscoped query", + coverage_exit: "1", + ) + compiler_path = File.join(project_root, "fake-tools", "crystal-alpha") + File.write(compiler_path, "#!/bin/sh\nexit 0\n") + File.chmod(compiler_path, 0o755) + payload = {"tool_name" => "Edit", "tool_input" => {"file_path" => File.join(project_root, "src/app.cr")}}.to_json + + post = AgentHookSpecHelper.run_hook(fixture, "post", payload) + + post[0].should eq(2) + post[2].should contain("error: unscoped query") + end + end + it "still fails when amber-lsp reports failed coverage" do SpecHelper.within_temp_directory do |project_root| fixture = AgentHookSpecHelper.create_ready_project( diff --git a/src/amber_cli/templates/agent/amber-agent-hook b/src/amber_cli/templates/agent/amber-agent-hook index 3f75f2e..d2c046a 100644 --- a/src/amber_cli/templates/agent/amber-agent-hook +++ b/src/amber_cli/templates/agent/amber-agent-hook @@ -710,6 +710,8 @@ AMBER_LOOKUP_GUIDANCE [ -z "$format_output" ] || printf '%s\n' "$format_output" diagnostics=$("$lsp" --check "$edited_file" 2>&1) diagnostic_status=$? + # A plain Crystal project has no Amber rules to check; formatting is its post-edit step. + [ "$diagnostics" = 'amber-lsp: declined project is not an Amber V2 stack project' ] && continue if [ "$diagnostic_status" -ne 0 ]; then [ -z "$diagnostics" ] || printf '%s\n' "$diagnostics" >&2 exit 2 From 1b4cb41d0ee861d15aab490d33daf8a639895f6a Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 09:55:15 -0400 Subject: [PATCH 62/69] feat(lsp): print usage for amber-lsp --help and each subcommand amber-lsp with no subcommand runs the language server on stdin, so --help used to wait silently, lookup --help failed, and context --help raised. Each now prints its usage; lookup lists its exit codes. --- spec/amber_lsp/lookup/lookup_command_spec.cr | 9 +++++++++ spec/amber_lsp/version_spec.cr | 8 ++++++++ src/amber_lsp.cr | 5 ++++- .../print_detected_rule_pack_contexts.cr | 8 +++++++- src/amber_lsp/lookup/lookup_command.cr | 20 +++++++++++++++++++ src/amber_lsp/version.cr | 14 +++++++++++++ 6 files changed, 62 insertions(+), 2 deletions(-) diff --git a/spec/amber_lsp/lookup/lookup_command_spec.cr b/spec/amber_lsp/lookup/lookup_command_spec.cr index faee639..23fd548 100644 --- a/spec/amber_lsp/lookup/lookup_command_spec.cr +++ b/spec/amber_lsp/lookup/lookup_command_spec.cr @@ -89,6 +89,15 @@ describe AmberLSP::Lookup::RunLookupCommand do end end + it "prints its usage and exit codes for --help" do + result = run_lookup_cli(["--help"], lookup_cli_cache_root) + + result[0].should eq(0) + result[1].should contain("Usage: amber-lsp lookup QUERY") + result[1].should contain("Exit codes: 0 found, 3 several candidates, 4 unknown, 5 absent, 2 failed.") + result[2].should eq("") + end + it "uses exit code 2 when the compiler cannot run" do with_lookup_cli_project do |project| stdout = IO::Memory.new diff --git a/spec/amber_lsp/version_spec.cr b/spec/amber_lsp/version_spec.cr index f22250e..e758561 100644 --- a/spec/amber_lsp/version_spec.cr +++ b/spec/amber_lsp/version_spec.cr @@ -1,6 +1,14 @@ require "./spec_helper" describe AmberLSP do + describe "USAGE" do + it "names every command-line mode" do + ["lookup QUERY", "hint", "--check FILE.cr", "context", "--version"].each do |mode| + AmberLSP::USAGE.should contain("amber-lsp #{mode}") + end + end + end + describe ".version_line" do it "names the server version, the amber_cli release, and the build commit" do AmberLSP.version_line.should eq( diff --git a/src/amber_lsp.cr b/src/amber_lsp.cr index f9ff43d..a3155af 100644 --- a/src/amber_lsp.cr +++ b/src/amber_lsp.cr @@ -48,7 +48,10 @@ require "./amber_lsp/lookup/verify_api_query" require "./amber_lsp/lookup/lookup_command" require "./amber_lsp/lookup/hint_command" -if ARGV.includes?("--version") +if ARGV.first?.in?("--help", "-h", "help") + STDOUT.puts AmberLSP::USAGE + exit 0 +elsif ARGV.includes?("--version") STDOUT.puts AmberLSP.version_line exit 0 elsif ARGV.first? == "lookup" diff --git a/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr b/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr index dd4d7f0..c88ef64 100644 --- a/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr +++ b/src/amber_lsp/library_rule_packs/print_detected_rule_pack_contexts.cr @@ -3,12 +3,18 @@ require "yaml" module AmberLSP::LibraryRulePacks class PrintDetectedRulePackContexts - Log = ::Log.for(self) + Log = ::Log.for(self) + USAGE = "Usage: amber-lsp context [--root DIR]\nPrints the library rule packs that apply to this project and the mode each one detected." def initialize(@list_of_arguments : Array(String)) end def perform : Int32 + if @list_of_arguments.includes?("--help") || @list_of_arguments.includes?("-h") + STDOUT.puts(USAGE) + return 0 + end + project_context = AmberLSP::ProjectContext.detect(project_root_from_arguments) list_of_rule_packs = LoadRulePacksForProject.new(project_context).load_rule_packs list_of_output_lines = [] of String diff --git a/src/amber_lsp/lookup/lookup_command.cr b/src/amber_lsp/lookup/lookup_command.cr index 138bbb1..4c00940 100644 --- a/src/amber_lsp/lookup/lookup_command.cr +++ b/src/amber_lsp/lookup/lookup_command.cr @@ -71,6 +71,21 @@ module AmberLSP::Lookup end class RunLookupCommand + USAGE = <<-TEXT + Usage: amber-lsp lookup QUERY [--root DIR] [--json] [--verify] + amber-lsp lookup --at FILE:LINE:COL [--root DIR] [--json] [--verify] + + Looks up a Crystal API in the project, its shards in lib/, and the standard library. + QUERY is Type.method (class method), Type#method (instance method), or Type. + + --root DIR project root (default: the current directory) + --at LOC look up the call at FILE:LINE:COL instead of a QUERY + --json print the answer as JSON + --verify compile a probe when the answer is unknown, to confirm the return type + + Exit codes: 0 found, 3 several candidates, 4 unknown, 5 absent, 2 failed. + TEXT + def initialize( @arguments : Array(String), @cache_root : String = APIIndexCache.default_root, @@ -81,6 +96,11 @@ module AmberLSP::Lookup end def perform : Int32 + if @arguments.includes?("--help") || @arguments.includes?("-h") + @stdout.puts(USAGE) + return 0 + end + options = parse_options query = lookup_query(options) card_collection = LoadAPICards.new(options.root_path).perform diff --git a/src/amber_lsp/version.cr b/src/amber_lsp/version.cr index 5577aa5..78d9679 100644 --- a/src/amber_lsp/version.cr +++ b/src/amber_lsp/version.cr @@ -8,6 +8,20 @@ module AmberLSP # The git commit the binary was built from, or "unknown" outside a git checkout. BUILD_COMMIT = {{ `git -C "#{__DIR__}" rev-parse --short=12 HEAD 2>/dev/null || echo unknown`.chomp.stringify }} + # Printed by `amber-lsp --help`. With no arguments amber-lsp runs the language server on stdin. + USAGE = <<-TEXT + Usage: + amber-lsp run the language server over stdin and stdout (editors start this) + amber-lsp lookup QUERY look up a Crystal API: Type.method, Type#method, or Type + amber-lsp hint read compiler output on stdin and print matching API card hints + amber-lsp --check FILE.cr check one file with the Amber V2 rules + amber-lsp context [--root DIR] print the library rule packs detected for this project + amber-lsp --version print the build: version, amber_cli release, commit + + Add --help after lookup, hint, or context for its options. + Guide: https://github.com/amberframework/amber_cli/blob/main/docs/guides/ai-assistants.md + TEXT + # One line that identifies this exact build, printed by `amber-lsp --version`. def self.version_line : String "amber-lsp #{VERSION} (amber_cli #{RELEASE}, commit #{BUILD_COMMIT})" From daf310cdf8d36b254c77ed363efe76d4fa6ae0fd Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 09:57:34 -0400 Subject: [PATCH 63/69] fix(lookup): fall back to stock crystal when crystal-alpha is absent The Homebrew formula installs stock crystal, so every lookup failed with "crystal-alpha is not available on PATH". Stock crystal builds the same docs index; crystal-alpha stays preferred when installed. --- .../lookup/default_compiler_command_spec.cr | 34 +++++++++++++++++++ src/amber_lsp/lookup/api_cards.cr | 6 ++-- .../lookup/build_layered_api_index.cr | 7 ++-- .../lookup/default_compiler_command.cr | 7 ++++ src/amber_lsp/lookup/lookup_command.cr | 3 +- src/amber_lsp/lookup/verify_api_query.cr | 3 +- 6 files changed, 53 insertions(+), 7 deletions(-) create mode 100644 spec/amber_lsp/lookup/default_compiler_command_spec.cr create mode 100644 src/amber_lsp/lookup/default_compiler_command.cr diff --git a/spec/amber_lsp/lookup/default_compiler_command_spec.cr b/spec/amber_lsp/lookup/default_compiler_command_spec.cr new file mode 100644 index 0000000..6db6b7f --- /dev/null +++ b/spec/amber_lsp/lookup/default_compiler_command_spec.cr @@ -0,0 +1,34 @@ +require "../spec_helper" +require "../../../src/amber_lsp/lookup/default_compiler_command" + +private def with_compilers_on_path(list_of_compiler_names : Array(String), &) + directory = File.join(Dir.tempdir, "amber-lsp-compilers-#{Random::Secure.hex(6)}") + Dir.mkdir_p(directory) + list_of_compiler_names.each do |compiler_name| + compiler_path = File.join(directory, compiler_name) + File.write(compiler_path, "#!/bin/sh\nexit 0\n") + File.chmod(compiler_path, 0o755) + end + original_path = ENV["PATH"]? + ENV["PATH"] = directory + begin + yield + ensure + ENV["PATH"] = original_path + FileUtils.rm_rf(directory) + end +end + +describe "AmberLSP::Lookup.default_compiler_command" do + it "prefers crystal-alpha when it is installed" do + with_compilers_on_path(["crystal-alpha", "crystal"]) do + AmberLSP::Lookup.default_compiler_command.should eq("crystal-alpha") + end + end + + it "falls back to stock crystal when crystal-alpha is not installed" do + with_compilers_on_path(["crystal"]) do + AmberLSP::Lookup.default_compiler_command.should eq("crystal") + end + end +end diff --git a/src/amber_lsp/lookup/api_cards.cr b/src/amber_lsp/lookup/api_cards.cr index 02397c7..87a3860 100644 --- a/src/amber_lsp/lookup/api_cards.cr +++ b/src/amber_lsp/lookup/api_cards.cr @@ -3,6 +3,7 @@ require "json" require "yaml" require "./source_models" +require "./default_compiler_command" require "../cards/embedded_api_cards" module AmberLSP::Lookup @@ -384,11 +385,12 @@ module AmberLSP::Lookup end private def read_crystal_version : String + compiler_command = Lookup.default_compiler_command output = IO::Memory.new errors = IO::Memory.new - status = Process.run("crystal-alpha", ["--version"], output: output, error: errors) + status = Process.run(compiler_command, ["--version"], output: output, error: errors) unless status.success? - raise ArgumentError.new("crystal-alpha --version failed: #{errors.to_s.strip}") + raise ArgumentError.new("#{compiler_command} --version failed: #{errors.to_s.strip}") end version_match = output.to_s.match(/\bCrystal\s+(\d+\.\d+(?:\.\d+)?(?:-[0-9A-Za-z.-]+)?)/) diff --git a/src/amber_lsp/lookup/build_layered_api_index.cr b/src/amber_lsp/lookup/build_layered_api_index.cr index f579149..65f991a 100644 --- a/src/amber_lsp/lookup/build_layered_api_index.cr +++ b/src/amber_lsp/lookup/build_layered_api_index.cr @@ -6,6 +6,7 @@ require "yaml" require "./build_crystal_docs_json" require "./api_cards" +require "./default_compiler_command" require "./index_cache" require "./merge_api_index_layers" require "./normalize_crystal_docs" @@ -23,12 +24,12 @@ module AmberLSP::Lookup end class DetectCrystalAlpha - def initialize(@compiler_command : String = "crystal-alpha") + def initialize(@compiler_command : String = Lookup.default_compiler_command) end def perform : CrystalAlphaIdentity compiler_path = Process.find_executable(@compiler_command) - raise APIIndexBuildError.new("crystal-alpha is not available on PATH") unless compiler_path + raise APIIndexBuildError.new("#{@compiler_command} is not available on PATH") unless compiler_path output = run_compiler_command(compiler_path, ["--version"]) version = output.strip @@ -72,7 +73,7 @@ module AmberLSP::Lookup @root_path : String, @cache_root : String = APIIndexCache.default_root, @list_of_docs_flags_by_library : Hash(String, Array(String)) = {} of String => Array(String), - @compiler_command : String = "crystal-alpha", + @compiler_command : String = Lookup.default_compiler_command, ) @root_path = File.expand_path(@root_path) end diff --git a/src/amber_lsp/lookup/default_compiler_command.cr b/src/amber_lsp/lookup/default_compiler_command.cr new file mode 100644 index 0000000..cc1e5c2 --- /dev/null +++ b/src/amber_lsp/lookup/default_compiler_command.cr @@ -0,0 +1,7 @@ +module AmberLSP::Lookup + # crystal-alpha is preferred. Stock crystal builds the same docs index, so a + # Homebrew install that only has crystal still answers lookups. + def self.default_compiler_command : String + Process.find_executable("crystal-alpha") ? "crystal-alpha" : "crystal" + end +end diff --git a/src/amber_lsp/lookup/lookup_command.cr b/src/amber_lsp/lookup/lookup_command.cr index 4c00940..28f7008 100644 --- a/src/amber_lsp/lookup/lookup_command.cr +++ b/src/amber_lsp/lookup/lookup_command.cr @@ -5,6 +5,7 @@ require "./build_layered_api_index" require "./resolve_api_query" require "./verify_api_query" require "./extract_lookup_query_at_position" +require "./default_compiler_command" module AmberLSP::Lookup struct LookupCLIOptions @@ -89,7 +90,7 @@ module AmberLSP::Lookup def initialize( @arguments : Array(String), @cache_root : String = APIIndexCache.default_root, - @compiler_command : String = "crystal-alpha", + @compiler_command : String = Lookup.default_compiler_command, @stdout : IO = STDOUT, @stderr : IO = STDERR, ) diff --git a/src/amber_lsp/lookup/verify_api_query.cr b/src/amber_lsp/lookup/verify_api_query.cr index 6efd190..fb542eb 100644 --- a/src/amber_lsp/lookup/verify_api_query.cr +++ b/src/amber_lsp/lookup/verify_api_query.cr @@ -4,6 +4,7 @@ require "path" require "random/secure" require "set" require "yaml" +require "./default_compiler_command" require "./index_models" require "./index_cache" @@ -35,7 +36,7 @@ module AmberLSP::Lookup @project_root_path : String, @query : String, @resolution : APIResolution, - @compiler_command : String = "crystal-alpha", + @compiler_command : String = Lookup.default_compiler_command, ) @project_root_path = File.expand_path(@project_root_path) end From f6df05175c5f3690a05664023d4ebbddb847d870 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 09:58:47 -0400 Subject: [PATCH 64/69] docs: explain agent tooling for Amber, plain Crystal apps, and shards The README's LSP and agent sections assumed an Amber project. One section now says the tooling works in any Crystal project and links a full guide: install, setup and trust, the committed files, what each hook does, every amber-lsp command with exit codes, API cards, and doctor fixes. --- README.md | 76 ++++++-------------- docs/guides/ai-assistants.md | 135 ++++++++++++++++++++++++++++++----- 2 files changed, 140 insertions(+), 71 deletions(-) diff --git a/README.md b/README.md index 07a2f0b..eb33ee9 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,9 @@ Amber CLI is the standalone command-line companion for Amber V2. CLI `2.0.7` creates the supported Amber `2.0.0-beta.5` ECR web application and includes -development, generator, database, and LSP tooling. +development, generator, database, and LSP tooling. Its agent tooling also +works in plain Crystal apps and shards; see +[Agent tooling](#agent-tooling-for-any-crystal-project). Amber V2 is a beta. The release-gated path is a web application on Apple Silicon macOS, x86_64 Linux, or ARM64 Linux. Windows x86-64 passes the complete @@ -159,8 +161,8 @@ SQL migration to `db/migrations/`, and the resource route to `config/routes.cr`. | `amber assets build` | Supported | Fingerprint `app/assets/` into generated `public/assets/` output | | `amber assets check` | Supported | Verify manifest, bytes, integrity, MIME, and compressed output without changing it | | `amber new APP --type native` | Preview | Not part of the beta platform guarantee | -| `amber setup:lsp` | Available | Register the project-local Claude Amber LSP plugin | -| `amber setup:agent` (`amber agent`) | Available | Install Claude Code and Codex hooks and setup metadata | +| `amber setup:lsp` | Available | Register only the project-local Claude `amber-lsp` plugin | +| `amber setup:agent` (`amber agent`) | Available | Set up Claude Code and Codex in any Crystal project | | `amber doctor` | Available | Check LSP readiness, hook trust, ignore rules, and API lookup status | Run `amber --help` or `amber COMMAND --help` for command syntax. The detailed @@ -185,66 +187,34 @@ Report CLI, template, or binary problems at `crystal --version`, `amber --version`, install method, command, and complete output. -## LSP +## Agent tooling for any Crystal project -The release archive includes `amber-lsp`. From an Amber project: +`amber-lsp` and `amber setup:agent` help Claude Code and Codex write Crystal +that compiles the first time. They work in Amber V2 apps, plain Crystal apps, +and Crystal shards. From the project root: ```bash -amber setup:lsp +amber setup:agent # install Claude Code and Codex hooks and the amber-lsp plugin +amber doctor # confirm the setup, binary checksum, and agent trust ``` -This writes a tracked directory marketplace under -`.amber/claude-marketplace/` and merges its registration into -`.claude/settings.json`, keeping the project's existing settings. Claude Code -loads the LSP server from that plugin after the user trusts the project folder. - -See the [LSP setup guide](https://github.com/amberframework/amber/blob/v2.0.0-beta.5/docs/guides/lsp-setup.md). - -## AI agent setup - -Run these commands in an Amber project to install or refresh the agent hooks -and check their setup: - -```bash -amber setup:agent -amber doctor -``` - -`amber new` runs `amber setup:agent` automatically. Run it again after -upgrading Amber CLI to refresh `.amber/agent_setup.json`, -`.amber/amber-agent-hook`, `.claude/settings.json`, `.codex/hooks.json`, and the -Claude plugin files under `.amber/claude-marketplace/`. Existing hook settings -and instructions are merged, so setup is safe to repeat. The generated -SessionStart hook reports whether setup is ready; when setup is incomplete, it -asks the agent to stop and have the user run `amber setup:agent` before editing -Crystal files. The pre-tool hook blocks Crystal writes until the `amber-lsp` -binary meets the recorded minimum version, its checksum matches, and the LSP -covers the project's main entry file. Install the binary with its adjacent -`.sha256` file or the release `checksums.txt` under `share/amber_cli/`. - -Run `amber doctor` to check hook files, Claude workspace trust and plugin -status, Codex project hook trust, Git ignore rules that would hide hooks from -worktrees, and API lookup index freshness. Doctor reads the agent trust records -without changing them. Claude and Codex may need the project and its hooks -trusted before their project-level hooks can run. - -Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints. - -Use `amber-lsp` to look up an API or get a hint for compiler output: +The agent then looks up each library method before calling it: ```bash -amber-lsp lookup 'Dir.mkdir_p' -crystal-alpha build src/my_app.cr 2>&1 | amber-lsp hint +amber-lsp lookup 'Dir.mkdir_p' # class method +amber-lsp lookup 'String#split' # instance method +crystal build src/my_app.cr 2>&1 | amber-lsp hint ``` -See the [AI assistant setup guide](docs/guides/ai-assistants.md) for the short -project setup steps. +Lookup answers from the project's own code, every shard in `lib/`, and the +Crystal standard library. In Amber V2 apps, edits are also checked against the +Amber rules. `amber new` runs `setup:agent` automatically; run it again after +upgrading. Commit the generated `.amber/`, `.claude/settings.json`, and +`.codex/hooks.json` so clones and worktrees share the setup. -The post-edit hook formats and checks only the changed Crystal file with -`amber-lsp --check`; the stop hook asks -`crystal-alpha watch build` for compiler feedback, or checks the target in -`shard.yml` with `--no-codegen` when no watcher is running. Install -`crystal-alpha` and checksum-verified `amber-lsp` for the full loop. +The [agent tooling guide](docs/guides/ai-assistants.md) covers trust prompts, +what each hook does, every `amber-lsp` command, API cards for library authors, +and troubleshooting. ## Contributing diff --git a/docs/guides/ai-assistants.md b/docs/guides/ai-assistants.md index 16823c4..862eec6 100644 --- a/docs/guides/ai-assistants.md +++ b/docs/guides/ai-assistants.md @@ -1,32 +1,131 @@ -# AI assistant setup +# Agent tooling for Crystal projects -Amber CLI adds project hooks for Claude Code and Codex. From the project root, -run: +Amber CLI ships two tools that help coding agents such as Claude Code and Codex +write Crystal that compiles the first time: + +- **`amber-lsp`** answers API questions from your project's real code: its + own source, every shard in `lib/`, and the Crystal standard library. It also + runs as a language server for editors. +- **`amber setup:agent`** connects Claude Code and Codex to `amber-lsp` in one + project. It installs hooks that tell the agent to look methods up before it + calls them, and that hold back Crystal edits until the tooling is ready. + +Both work in any Crystal project. You do not need an Amber app. + +| Project | API lookup and compiler hints | Edit gate and formatting | Amber V2 rule checks | +|---|---|---|---| +| Amber V2 app | Yes | Yes | Yes, plus library rule packs | +| Plain Crystal app (`crystal init app`) | Yes | Yes | Not applicable | +| Crystal shard (`crystal init lib`) | Yes | Yes | Not applicable | + +## Install + +```bash +HOMEBREW_NO_AUTO_UPDATE=1 brew install amberframework/amber_cli/amber_cli +amber --version +amber-lsp --version +``` + +Homebrew installs `amber`, `amber-lsp`, the checksum file that the hooks +verify `amber-lsp` against, the bundled Crystal API card, Crystal, and +Minecart. See the [README](../../README.md#install) for the tap trust step and +for direct release archives. + +`amber-lsp` uses `crystal-alpha` when it is installed and stock `crystal` +otherwise. + +## Set up a project + +From the project root (the directory that holds `shard.yml`): ```bash amber setup:agent amber doctor ``` -`setup:agent` creates or refreshes the hook files and merges them with the -project's existing Claude and Codex settings. Review and commit the generated -files. `amber doctor` reports whether the hooks, trust settings, LSP binary, -and API lookup index are ready. +`amber new` runs `setup:agent` for you. Run it again after you upgrade Amber +CLI, because it refreshes the generated files. It merges with your existing +settings and instructions, so it is safe to repeat. + +Then trust the project once in each agent: + +- **Claude Code:** open the project and accept the workspace trust prompt. + Project hooks and the bundled `amber-lsp` plugin load only in trusted + folders. +- **Codex:** review and trust the project hooks in `.codex/hooks.json`. Codex + skips project hooks it has not trusted. -Install a checksum-verified `amber-lsp` binary and `crystal-alpha` to enable -Crystal edit checks. The pre-tool hook checks the edited Crystal file before -the agent writes it, and the post-edit hook formats and analyzes that file. -If setup is incomplete, run `amber setup:agent` again and follow the readiness -message. +`amber doctor` reports both trust states, and it ends with exit 0 when +everything is ready. -Before writing code that calls Grant, Amber, asset_pipeline, or Crystal standard library methods, list each method you will call and run `amber-lsp lookup 'Type.method'` (class method) or `amber-lsp lookup 'Type#method'` (instance method) for each one, adding `--verify` when the answer is unknown. Use exactly the signatures and return types it reports, and follow any card note it prints. +### Commit the generated files -Use API lookup and compiler hints from the project root: +Commit these so every clone and every `git worktree` gets the same setup: + +| File | Purpose | +|---|---| +| `.amber/agent_setup.json` | The amber_cli version and the minimum `amber-lsp` version this setup expects | +| `.amber/amber-agent-hook` | The POSIX shell hook that Claude Code and Codex both call | +| `.amber/claude-marketplace/` | A project-local Claude plugin that registers `amber-lsp` as the Crystal language server | +| `.claude/settings.json` | Claude Code hooks, plus the plugin marketplace entry | +| `.codex/hooks.json` | Codex hooks | +| `CLAUDE.md`, `AGENTS.md` | An "Agent loop" block that tells the agent to look up library methods before calling them | + +When `.gitignore` or `.git/info/exclude` hides `.claude` or `.codex`, +`setup:agent` warns you: agent worktrees would run without the hooks. + +## What the agent experiences + +| When | What happens | +|---|---| +| Session start | The agent reads whether setup is ready, the `amber-lsp` version, and the instruction to look up methods first. | +| Before a `.cr` edit | The edit is refused if `amber-lsp` is missing, older than the recorded minimum, or fails its checksum. The refusal tells the agent to ask you to run `amber setup:agent`. | +| After a `.cr` edit | The file is formatted. In an Amber V2 app it is also checked against the Amber rules, and findings go back to the agent. | +| When the agent stops | The project is built (`--no-codegen`, or through `crystal-alpha watch` when a watcher is running), and compiler errors go back to the agent. | + +## Use amber-lsp yourself + +Run these from the project root, or pass `--root DIR`. ```bash -amber-lsp lookup 'Dir.mkdir_p' -crystal-alpha build src/my_app.cr 2>&1 | amber-lsp hint +amber-lsp lookup 'Dir.mkdir_p' # a class method +amber-lsp lookup 'String#split' # an instance method +amber-lsp lookup 'MyApp::Invoice' # a type +amber-lsp lookup 'Array#map' --json # JSON for scripts +crystal build src/my_app.cr 2>&1 | amber-lsp hint +amber-lsp --help ``` -`lookup` searches the Amber and Crystal API index. `hint` reads compiler output -from standard input and prints a matching API card hint when one is available. +`lookup` prints the signature, the resolved return type, where the method is +defined, its doc comment, and any notes from API cards. In a script, read the +exit code: 0 found, 3 several candidates, 4 unknown, 5 absent, 2 failed. +`--verify` compiles a small probe when the index cannot answer, which confirms +whether the method exists and what it returns. + +`hint` reads compiler output on stdin. When an error matches a known mistake +described in an API card, it prints the fix and an example. + +The first lookup in a project builds a docs index for each layer (the project, +each shard, and the standard library) and caches it under +`~/.cache/amber-lsp/index`. Later lookups take milliseconds, and a layer is +rebuilt when its source changes. + +## For library authors + +A shard can ship notes and compiler-error hints for its own API in +`.amber-lsp/api/*.yml`. Every project that depends on the shard picks the card +up from `lib/`. See [API cards](../api-cards.md) for the format, and +[rule packs](../rule-packs.md) for edit-time checks a library can add, which +run in Amber V2 apps. + +## Troubleshooting + +Run `amber doctor`. Each failed item names its fix. The common ones: + +| Doctor says | Do this | +|---|---| +| `amber-lsp` was not found | Install Amber CLI, or set `AMBER_LSP_BIN` to the binary. | +| The checksum is missing or does not match | Reinstall from Homebrew or a release archive; keep `amber-lsp.sha256` next to the binary or `checksums.txt` in `share/amber_cli/`. | +| Claude workspace trust is not accepted | Open the project in Claude Code and accept the trust prompt. | +| Codex project hooks are not all trusted | Trust `.codex/hooks.json` in Codex. | +| Hooks or instructions are out of date | Run `amber setup:agent`. | From af34c36ceeb583117e887d2076f45f5d416baaeb Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 10:00:29 -0400 Subject: [PATCH 65/69] build(ci): pin Minecart 2025.11.25.7 at 091e8e2 The release candidate now merges Minecart's alpha branch (2025.11.25.6) and pins its own dependencies by commit, so its commit and tree changed. --- scripts/install_pinned_minecart.ps1 | 4 ++-- scripts/install_pinned_minecart.sh | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/scripts/install_pinned_minecart.ps1 b/scripts/install_pinned_minecart.ps1 index 894eb1b..cfda050 100644 --- a/scripts/install_pinned_minecart.ps1 +++ b/scripts/install_pinned_minecart.ps1 @@ -1,7 +1,7 @@ $ErrorActionPreference = "Stop" -$minecartCommit = "4fea47ce6561ea0612cf1e0cf6955d782ef27929" -$minecartTree = "e0a0214c14d71f1031d83b760d426b09509b5d9d" +$minecartCommit = "091e8e2da15a0a4885a40b174d5a561da0201c1b" +$minecartTree = "2d00f70352d6ff70b29b1037bc85a316a6a13117" $installRoot = Join-Path $env:RUNNER_TEMP "amber-pinned-minecart" New-Item -ItemType Directory -Force $installRoot | Out-Null diff --git a/scripts/install_pinned_minecart.sh b/scripts/install_pinned_minecart.sh index 814594c..ae64486 100755 --- a/scripts/install_pinned_minecart.sh +++ b/scripts/install_pinned_minecart.sh @@ -2,8 +2,8 @@ set -euo pipefail # The source commit is verified by its Git tree before it can run the web smoke. -minecart_commit=4fea47ce6561ea0612cf1e0cf6955d782ef27929 -minecart_tree=e0a0214c14d71f1031d83b760d426b09509b5d9d +minecart_commit=091e8e2da15a0a4885a40b174d5a561da0201c1b +minecart_tree=2d00f70352d6ff70b29b1037bc85a316a6a13117 minecart_repository="${MINECART_SOURCE_REPOSITORY:-https://github.com/crimson-knight/shards.git}" minecart_source_ref="${MINECART_SOURCE_REF:-refs/tags/v2025.11.25.7}" install_root="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/amber-pinned-minecart" From 9861c9f0e2d9981f92e9e46cc4d2bff8856bcf90 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 10:48:05 -0400 Subject: [PATCH 66/69] build(ci): pin Minecart 2025.11.25.8 and build it with stock crystal CI runners and Homebrew have stock crystal only, so the installer now uses crystal-alpha when present and crystal otherwise. Minecart .8 compiles on Windows, which the generated-app gate needs. --- scripts/install_pinned_minecart.ps1 | 6 +++--- scripts/install_pinned_minecart.sh | 16 ++++++++++------ 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/scripts/install_pinned_minecart.ps1 b/scripts/install_pinned_minecart.ps1 index cfda050..486da0f 100644 --- a/scripts/install_pinned_minecart.ps1 +++ b/scripts/install_pinned_minecart.ps1 @@ -1,7 +1,7 @@ $ErrorActionPreference = "Stop" -$minecartCommit = "091e8e2da15a0a4885a40b174d5a561da0201c1b" -$minecartTree = "2d00f70352d6ff70b29b1037bc85a316a6a13117" +$minecartCommit = "0ffac921c68774ea143aeaa83c369d0a38c2d3e5" +$minecartTree = "f0e8a69bffe22afd0cee9b24994bde629384f093" $installRoot = Join-Path $env:RUNNER_TEMP "amber-pinned-minecart" New-Item -ItemType Directory -Force $installRoot | Out-Null @@ -24,7 +24,7 @@ try { & crystal build src/shards.cr -o bin/minecart.exe if ($LASTEXITCODE -ne 0) { throw "Minecart build failed" } $minecartVersion = (& ./bin/minecart.exe --version).Trim() - if ($minecartVersion -notlike "Minecart 2025.11.25.7*") { + if ($minecartVersion -notlike "Minecart 2025.11.25.8*") { throw "Built Minecart version did not match the pin: $minecartVersion" } } finally { diff --git a/scripts/install_pinned_minecart.sh b/scripts/install_pinned_minecart.sh index ae64486..ccbdcac 100755 --- a/scripts/install_pinned_minecart.sh +++ b/scripts/install_pinned_minecart.sh @@ -2,10 +2,10 @@ set -euo pipefail # The source commit is verified by its Git tree before it can run the web smoke. -minecart_commit=091e8e2da15a0a4885a40b174d5a561da0201c1b -minecart_tree=2d00f70352d6ff70b29b1037bc85a316a6a13117 +minecart_commit=0ffac921c68774ea143aeaa83c369d0a38c2d3e5 +minecart_tree=f0e8a69bffe22afd0cee9b24994bde629384f093 minecart_repository="${MINECART_SOURCE_REPOSITORY:-https://github.com/crimson-knight/shards.git}" -minecart_source_ref="${MINECART_SOURCE_REF:-refs/tags/v2025.11.25.7}" +minecart_source_ref="${MINECART_SOURCE_REF:-refs/tags/v2025.11.25.8}" install_root="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/amber-pinned-minecart" mkdir -p "$install_root" @@ -20,11 +20,15 @@ git -C "$install_root" checkout -q --detach FETCH_HEAD test "$(git -C "$install_root" rev-parse HEAD)" = "$minecart_commit" test "$(git -C "$install_root" rev-parse 'HEAD^{tree}')" = "$minecart_tree" -make -C "$install_root" bin/minecart bin/shards-alpha CRYSTAL="${CRYSTAL:-crystal-alpha}" +# crystal-alpha is preferred; CI runners and Homebrew have stock crystal. +if [[ -z "${CRYSTAL:-}" ]]; then + if command -v crystal-alpha >/dev/null 2>&1; then CRYSTAL=crystal-alpha; else CRYSTAL=crystal; fi +fi +make -C "$install_root" bin/minecart bin/shards-alpha CRYSTAL="$CRYSTAL" test -x "$install_root/bin/minecart" test -x "$install_root/bin/shards-alpha" -"$install_root/bin/minecart" --version | grep -F "Minecart 2025.11.25.7" -"$install_root/bin/shards-alpha" --version | grep -F "Minecart 2025.11.25.7" +"$install_root/bin/minecart" --version | grep -F "Minecart 2025.11.25.8" +"$install_root/bin/shards-alpha" --version | grep -F "Minecart 2025.11.25.8" if [[ -n "${GITHUB_PATH:-}" ]]; then echo "$install_root/bin" >> "$GITHUB_PATH" From 9690dfb6fd936503ea1b885027aa82116b62299d Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 10:53:50 -0400 Subject: [PATCH 67/69] ci: run lookup specs and the beta contract on stock Crystal runners The lookup spec helper named crystal-alpha, which CI runners do not have; it now uses the same compiler fallback as amber-lsp. The beta contract script still read commands/new.cr, renamed to new_command.cr. --- scripts/check_beta_contract.sh | 22 ++++++++++---------- spec/amber_lsp/lookup/lookup_command_spec.cr | 2 +- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/scripts/check_beta_contract.sh b/scripts/check_beta_contract.sh index 1f0d033..396fcf9 100755 --- a/scripts/check_beta_contract.sh +++ b/scripts/check_beta_contract.sh @@ -6,14 +6,14 @@ cli_version="$(sed -n 's/.*VERSION = "\([^"]*\)".*/\1/p' src/amber_cli.cr | head test "$shard_version" = "2.0.7" test "$cli_version" = "$shard_version" -grep -F 'github: amberframework/amber' src/amber_cli/commands/new.cr -grep -F 'version: 2.0.0-beta.5' src/amber_cli/commands/new.cr -grep -F 'template: ecr' src/amber_cli/commands/new.cr -grep -F 'model: grant' src/amber_cli/commands/new.cr -grep -F 'database: #{database}' src/amber_cli/commands/new.cr -grep -F 'github: crimson-knight/grant' src/amber_cli/commands/new.cr -grep -F 'github: amberframework/asset_pipeline' src/amber_cli/commands/new.cr -grep -F 'version: 0.37.0' src/amber_cli/commands/new.cr +grep -F 'github: amberframework/amber' src/amber_cli/commands/new_command.cr +grep -F 'version: 2.0.0-beta.5' src/amber_cli/commands/new_command.cr +grep -F 'template: ecr' src/amber_cli/commands/new_command.cr +grep -F 'model: grant' src/amber_cli/commands/new_command.cr +grep -F 'database: #{database}' src/amber_cli/commands/new_command.cr +grep -F 'github: crimson-knight/grant' src/amber_cli/commands/new_command.cr +grep -F 'github: amberframework/asset_pipeline' src/amber_cli/commands/new_command.cr +grep -F 'version: 0.37.0' src/amber_cli/commands/new_command.cr grep -F 'github: amberframework/asset_pipeline' src/amber_cli/templates/app/shard.yml.ecr grep -F 'version: 0.37.0' src/amber_cli/templates/app/shard.yml.ecr grep -F 'github: amberframework/amber' src/amber_cli/generators/native_app.cr @@ -48,8 +48,8 @@ grep -F '[string]$FrameworkRepository = "amberframework/amber"' scripts/smoke_ge grep -F '" github: $FrameworkRepository"' scripts/smoke_generated_web.ps1 test -s src/amber_cli/templates/app/config/database.cr.ecr test -s src/amber_cli/templates/app/config/assets.cr.ecr -grep -F 'Your new idea' src/amber_cli/commands/new.cr -grep -F -- '--amber-accent: #e96918' src/amber_cli/commands/new.cr +grep -F 'Your new idea' src/amber_cli/commands/new_command.cr +grep -F -- '--amber-accent: #e96918' src/amber_cli/commands/new_command.cr grep -F 'Your new idea' src/amber_cli/templates/app/src/views/home/index.ecr.ecr test -s src/amber_cli/templates/app/app/assets/stylesheets/app.css test -s src/amber_cli/templates/app/app/assets/javascript/app.js @@ -81,7 +81,7 @@ if grep -Ein 'amberframework/amber-cli|brew install amber-cli|brew install amber exit 1 fi -if grep -Eir 'crimson-knight/(amber|gemma)' src/amber_cli/templates/app src/amber_cli/commands/new.cr; then +if grep -Eir 'crimson-knight/(amber|gemma)' src/amber_cli/templates/app src/amber_cli/commands/new_command.cr; then echo "supported web template contains a personal Amber or Gemma dependency" >&2 exit 1 fi diff --git a/spec/amber_lsp/lookup/lookup_command_spec.cr b/spec/amber_lsp/lookup/lookup_command_spec.cr index 23fd548..abdefab 100644 --- a/spec/amber_lsp/lookup/lookup_command_spec.cr +++ b/spec/amber_lsp/lookup/lookup_command_spec.cr @@ -162,7 +162,7 @@ end private def run_lookup_cli(arguments : Array(String), cache_root : String) : Tuple(Int32, String, String) stdout = IO::Memory.new stderr = IO::Memory.new - code = AmberLSP::Lookup::RunLookupCommand.new(arguments, cache_root, "crystal-alpha", stdout, stderr).perform + code = AmberLSP::Lookup::RunLookupCommand.new(arguments, cache_root, AmberLSP::Lookup.default_compiler_command, stdout, stderr).perform {code, stdout.to_s, stderr.to_s} end From 8b871260f689d3ab652cd51978ae29e292ce2c63 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 11:01:09 -0400 Subject: [PATCH 68/69] ci: test with Homebrew Crystal on macOS and the installed compiler version The macOS install-crystal tarball has no llvm-config, so lookup could not build standard library docs there; Homebrew's Crystal, which users install, does. Two specs pinned Crystal 1.21.0 and broke when runners moved to 1.21.1; they now read the installed compiler's version. --- .github/workflows/ci.yml | 6 ++++++ spec/amber_lsp/lookup/api_cards_spec.cr | 9 ++++++++- spec/amber_lsp/lookup/hint_command_spec.cr | 9 ++++++++- 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e50a167..2a21a2f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -28,10 +28,16 @@ jobs: run: sudo apt-get update - name: Install Crystal + if: runner.os == 'Linux' uses: crystal-lang/install-crystal@d8ef131ecec0352ce0e39b81b0a6d95def58fe2f # v1 with: crystal: ${{ matrix.crystal }} + - name: Install Crystal with Homebrew + # Homebrew's Crystal brings llvm-config, which the API lookup specs need to + # build the standard library docs; the install-crystal macOS tarball has none. + if: runner.os == 'macOS' + run: brew install crystal - name: Cache shards uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4 diff --git a/spec/amber_lsp/lookup/api_cards_spec.cr b/spec/amber_lsp/lookup/api_cards_spec.cr index 2428c2d..89a161a 100644 --- a/spec/amber_lsp/lookup/api_cards_spec.cr +++ b/spec/amber_lsp/lookup/api_cards_spec.cr @@ -119,7 +119,7 @@ describe AmberLSP::Lookup::LoadAPICards do loaded_card.should_not be_nil if loaded = loaded_card loaded.source_path.should eq(File.realpath(card_path)) - loaded.resolved_version.should eq("1.21.0") + loaded.resolved_version.should eq(installed_crystal_version) loaded.origin.should eq(AmberLSP::Lookup::APICardOrigin::Bundled) else fail("expected the installed Crystal API card") @@ -231,3 +231,10 @@ private def write_test_card(project : String, library : String, pattern : String Dir.mkdir_p(api_path) File.write(File.join(api_path, "#{library}.yml"), card_yaml(library, pattern, hint)) end + +# The version the installed compiler reports; cards resolve against it. +private def installed_crystal_version : String + output = IO::Memory.new + Process.run(AmberLSP::Lookup.default_compiler_command, ["--version"], output: output) + output.to_s[/Crystal\s+(\d+\.\d+\.\d+)/, 1] +end diff --git a/spec/amber_lsp/lookup/hint_command_spec.cr b/spec/amber_lsp/lookup/hint_command_spec.cr index b6267fe..f4595e7 100644 --- a/spec/amber_lsp/lookup/hint_command_spec.cr +++ b/spec/amber_lsp/lookup/hint_command_spec.cr @@ -89,7 +89,7 @@ describe AmberLSP::Lookup::RunHintCommand do code.should eq(0) output.to_s.lines.should eq([ - "hint: Crystal String has no `to_sym`; keep values as String, or call `to_s` when converting a Symbol to String. [crystal@1.21.0]", + "hint: Crystal String has no `to_sym`; keep values as String, or call `to_s` when converting a Symbol to String. [crystal@#{installed_crystal_version}]", " right: name.to_s", ]) end @@ -125,3 +125,10 @@ private def with_capture_hint_cli_project(&) yield project end end + +# The version the installed compiler reports; cards resolve against it. +private def installed_crystal_version : String + output = IO::Memory.new + Process.run(AmberLSP::Lookup.default_compiler_command, ["--version"], output: output) + output.to_s[/Crystal\s+(\d+\.\d+\.\d+)/, 1] +end From faed4b205063a84650b00e2dc02f01793f373527 Mon Sep 17 00:00:00 2001 From: crimson-knight Date: Tue, 6 Oct 2026 11:18:32 -0400 Subject: [PATCH 69/69] build(release): put shards on PATH from the pinned macOS Crystal archive The darwin archive keeps shards only in embedded/bin, so micrate's postinstall (shards build) failed during the frozen install. --- scripts/install_pinned_crystal.sh | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/scripts/install_pinned_crystal.sh b/scripts/install_pinned_crystal.sh index f36225c..3aed424 100755 --- a/scripts/install_pinned_crystal.sh +++ b/scripts/install_pinned_crystal.sh @@ -44,6 +44,11 @@ fi bin_directory="$(dirname "$compiler_path")" ln -sfn crystal "$bin_directory/crystal-alpha" +# The macOS archive keeps shards only in embedded/bin; dependency postinstall scripts call it. +if [[ ! -x "$bin_directory/shards" ]]; then + shards_path="$(find "$install_root" -path '*/bin/shards' -print -quit)" + [[ -n "$shards_path" ]] && ln -sfn "$shards_path" "$bin_directory/shards" +fi version_output="$("$bin_directory/crystal-alpha" --version)" if [[ "$version_output" != *"Crystal ${crystal_version} "* ]]; then printf 'Unexpected compiler version in verified archive: %s\n' "$version_output" >&2