From f41b5e50899265823f550a4953227e9e4d15e095 Mon Sep 17 00:00:00 2001 From: Hui Miao Date: Thu, 22 Oct 2026 08:19:02 +0800 Subject: [PATCH 01/10] feat: add azd ai agent config list-alpha Rebased onto the updated hosted agent feature flag branch. Squashes: - feat: add azd ai agent feature list - refactor: rename feature list to config list-alpha --- cli/azd/extensions/azure.ai.agents/README.md | 15 ++ .../docs/specs/preview-feature-flags.md | 11 +- .../azure.ai.agents/internal/cmd/config.go | 162 ++++++++++++++++++ .../internal/cmd/config_test.go | 159 +++++++++++++++++ .../internal/cmd/json_output.go | 11 +- .../azure.ai.agents/internal/cmd/root.go | 1 + .../azure.ai.agents/internal/cmd/root_test.go | 1 + .../cmd/testdata/TestAgentHelp-config.snap | 23 +++ .../TestAgentHelp-config_list-alpha.snap | 27 +++ .../cmd/testdata/TestAgentHelp-root.snap | 1 + .../TestAgentHelpPreviewsOff-root.snap | 1 + .../README.md | 1 + .../tier0/0.02-help-root.yaml | 2 +- .../tier0/0.32-config-list-alpha.yaml | 16 ++ 14 files changed, 426 insertions(+), 5 deletions(-) create mode 100644 cli/azd/extensions/azure.ai.agents/internal/cmd/config.go create mode 100644 cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go create mode 100644 cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config.snap create mode 100644 cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config_list-alpha.snap create mode 100644 cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml diff --git a/cli/azd/extensions/azure.ai.agents/README.md b/cli/azd/extensions/azure.ai.agents/README.md index 2f1c0bb6275..b815e34ea64 100644 --- a/cli/azd/extensions/azure.ai.agents/README.md +++ b/cli/azd/extensions/azure.ai.agents/README.md @@ -28,6 +28,21 @@ Opt-ins combine with OR. A false value never overrides a true value elsewhere. `alpha.all` and unrelated config keys, including `agents.hosted`, do not turn these features on. Ordinary hosted and plain prompt agents need no opt-in. +### Check feature status + +`azd ai agent config list-alpha` lists each feature with its name, description, +status in the selected environment, and environment variable, in the layout of +`azd config list-alpha`. Use `-e` to read another environment and +`--output json` for scripts. + +```bash +azd ai agent config list-alpha +azd ai agent config list-alpha -e dev --output json +``` + +When a status cannot be read, that entry shows `Unknown` with the error. The +command still lists every feature and then exits with an error. + While a feature is off, its flags, commands, and help sections are hidden from help. This covers `invoke --long-running`, `invoke --no-wait`, the `invocations` group, `init --voice`, and the `prompt-voice` choice of diff --git a/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md b/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md index dee7a5abc26..11f85314d91 100644 --- a/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md +++ b/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md @@ -17,6 +17,7 @@ Managed harness means `kind: prompt` with harness type `microsoft_copilot_previe azure.ai.agents/ ├── internal/ │ ├── cmd/ +│ │ ├── config.go # config list-alpha command │ │ ├── root.go # One pre-run check and the help hook │ │ └── preview_features.go # Annotations, enforcement, help hiding │ ├── project/ @@ -139,6 +140,12 @@ azd env set AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE true The [README](../../README.md#preview-feature-opt-ins) lists the equivalent commands for all three capabilities. +## Alpha feature listing + +`azd ai agent config list-alpha` mirrors `azd config list-alpha`. It prints every feature from `All` as a block with `Name`, `Description`, `Status` (`On` or `Off`), and `Environment variable`. It calls `IsEnabled` once per feature with the environment selected by `-e`, so the status matches what enforcement sees. It prints no source for an opt-in because the API returns only a decision. + +The default text output prints `Status: Unknown` and an `Error` line for a feature whose status cannot be read. `--output json` prints `{"features":[...]}` with `name`, `description`, `status`, `envVar`, and an `error` field when the read failed. Both formats list every feature before the command exits with an error that counts the failed reads. The command is read only and needs no authentication. + ## Documented enablement methods The documented ways to opt into a feature are listed below. These opt-ins are additive: any enabling value is sufficient. @@ -193,6 +200,6 @@ Disabled capabilities return the structured `feature_not_enabled` compatibility Tests cover source combinations, all supported boolean spellings and non-enabling values, nested per-feature alpha config, ignored `alpha.all` and unrelated keys, exact environment keys, explicit environment forwarding, optional scopes and sentinel compatibility, genuine failures despite positive sources, malformed responses, context forwarding, and snapshot isolation without input mutation. Registry tests cover declaration rules, the stable copy returned by `All`, and `Lookup`. -Command regressions run through the root command so the central pre-run applies. They cover which parsed flags and values require which feature, inherited command annotations, service gate placement, independent gates, obsolete hosted opt-in isolation, selected-environment forwarding for enforcement and help, help hiding per feature with a process fallback, hidden commands that stay reachable, staged manifests, local invocation, deployed-name fallback, unchanged inputs on rejection, ordinary flows, and retained read access. Help snapshots cover the full surface with every feature on and the default surface with every feature off. A walk over every command with every feature off rejects help text that names a disabled preview. +Command regressions run through the root command so the central pre-run applies. They cover which parsed flags and values require which feature, inherited command annotations, service gate placement, independent gates, obsolete hosted opt-in isolation, selected-environment forwarding for enforcement and help, help hiding per feature with a process fallback, hidden commands that stay reachable, staged manifests, local invocation, deployed-name fallback, unchanged inputs on rejection, ordinary flows, and retained read access. Help snapshots cover the full surface with every feature on and the default surface with every feature off. A walk over every command with every feature off rejects help text that names a disabled preview. Alpha listing tests cover text and JSON output, per-feature read errors, the selected environment, and the output flag allow list. -Run offline tests, build, vet, and scoped lint against the pinned SDK. Tier 0 scenarios cover the hidden and enabled help states. +Run offline tests, build, vet, and scoped lint against the pinned SDK. Tier 0 scenarios cover the hidden and enabled help states and `config list-alpha`. diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go new file mode 100644 index 00000000000..dd5b6b56faf --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go @@ -0,0 +1,162 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +package cmd + +import ( + "context" + "fmt" + "io" + "strings" + + "azureaiagent/internal/featureflags" + + "github.com/azure/azure-dev/cli/azd/pkg/azdext" + "github.com/spf13/cobra" +) + +// Status values match `azd config list-alpha`. +const ( + alphaStatusOn = "On" + alphaStatusOff = "Off" + alphaStatusUnknown = "Unknown" +) + +// newConfigCommand returns the `azd ai agent config` parent command. +func newConfigCommand(extCtx *azdext.ExtensionContext) *cobra.Command { + extCtx = ensureExtensionContext(extCtx) + cmd := &cobra.Command{ + Use: "config", + Short: "Inspect agent extension configuration.", + Long: `Inspect agent extension configuration. + +Use 'azd config' to change azd user configuration.`, + Example: ` # Display the agent features in alpha stage + azd ai agent config list-alpha`, + } + cmd.AddCommand(newConfigListAlphaCommand(extCtx)) + return cmd +} + +// AlphaFeatureItem is the JSON contract for one agent alpha feature. +// Fields added later are additive. Consumers must ignore unknown fields. +type AlphaFeatureItem struct { + // Name is the feature ID used under alpha in user config. + Name string `json:"name"` + // Description is a short summary of the feature. + Description string `json:"description,omitempty"` + // Status is On or Off for the selected environment, or Unknown when + // Error is set. + Status string `json:"status"` + // EnvVar is the environment variable that turns the feature on. + EnvVar string `json:"envVar"` + // Error is set when the status could not be read. + Error string `json:"error,omitempty"` +} + +type alphaFeatureListResponse struct { + Features []AlphaFeatureItem `json:"features"` +} + +type alphaFeatureReader func(context.Context, featureflags.Feature) (bool, error) + +func newConfigListAlphaCommand(extCtx *azdext.ExtensionContext) *cobra.Command { + cmd := &cobra.Command{ + Use: "list-alpha", + Short: "Display the list of agent features in alpha stage.", + Long: `Display the list of agent features in alpha stage with the status of each +in the selected environment and the environment variable that turns it on. + +Use -e/--environment to read another azd environment. When a status cannot be +read, the entry reports the error and the command exits with an error after +listing every feature.`, + Example: ` # Display the agent features in alpha stage + azd ai agent config list-alpha + + # Display the features for another environment as JSON + azd ai agent config list-alpha -e dev --output json + + # Turn on a specific alpha feature + azd config set alpha. on`, + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, args []string) error { + ctx := cmd.Context() + client, err := azdext.NewAzdClient() + if err != nil { + return fmt.Errorf("connecting to azd to read alpha features: %w", err) + } + defer client.Close() + environmentName := featureflags.Environment(ctx) + read := func(ctx context.Context, feature featureflags.Feature) (bool, error) { + return featureflags.IsEnabled(ctx, client, feature, environmentName) + } + return runConfigListAlpha(ctx, cmd.OutOrStdout(), extCtx.OutputFormat, read) + }, + } + azdext.RegisterFlagOptions(cmd, azdext.FlagOptions{ + Name: "output", + AllowedValues: []string{"json", "text"}, + Default: "text", + }) + return cmd +} + +// runConfigListAlpha reads every registered feature, writes the list, and +// then reports whether any status could not be read. +func runConfigListAlpha(ctx context.Context, out io.Writer, format string, read alphaFeatureReader) error { + features := featureflags.All() + items := make([]AlphaFeatureItem, 0, len(features)) + failed := 0 + for _, feature := range features { + item := AlphaFeatureItem{ + Name: feature.ID, + Description: feature.Description, + Status: alphaStatusOff, + EnvVar: feature.EnvVar, + } + enabled, err := read(ctx, feature) + switch { + case err != nil: + item.Status = alphaStatusUnknown + item.Error = err.Error() + failed++ + case enabled: + item.Status = alphaStatusOn + } + items = append(items, item) + } + + var err error + if format == "json" { + err = writeJSON(out, alphaFeatureListResponse{Features: items}) + } else { + err = writeAlphaFeatures(out, items) + } + if err != nil { + return err + } + if failed > 0 { + return fmt.Errorf("could not read the status of %d of %d alpha features", failed, len(items)) + } + return nil +} + +// writeAlphaFeatures writes one block per feature in the layout of +// `azd config list-alpha`, plus the environment variable. +func writeAlphaFeatures(out io.Writer, items []AlphaFeatureItem) error { + blocks := make([]string, 0, len(items)) + for _, item := range items { + lines := []string{ + "Name: " + item.Name, + "Description: " + item.Description, + "Status: " + item.Status, + "Environment variable: " + item.EnvVar, + } + if item.Error != "" { + lines = append(lines, "Error: "+item.Error) + } + blocks = append(blocks, strings.Join(lines, "\n")) + } + _, err := fmt.Fprintln(out, strings.Join(blocks, "\n\n")) + return err +} diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go new file mode 100644 index 00000000000..1fa78b8aaeb --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go @@ -0,0 +1,159 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "strings" + "testing" + + "azureaiagent/internal/featureflags" + + "github.com/azure/azure-dev/cli/azd/pkg/azdext" + "github.com/stretchr/testify/require" +) + +func TestConfigListAlphaText(t *testing.T) { + var out bytes.Buffer + err := runConfigListAlpha(t.Context(), &out, "text", + func(_ context.Context, feature featureflags.Feature) (bool, error) { + return feature == featureflags.Voice, nil + }) + require.NoError(t, err) + require.Equal(t, strings.Join([]string{ + "Name: agents.managed_harness", + "Description: Prompt agents that run on a Copilot harness", + "Status: Off", + "Environment variable: AZD_FOUNDRY_EXT_ENABLE_AGENTS_MANAGED_HARNESS", + "", + "Name: agents.voice", + "Description: Voice, prompt-voice, and hosted voice agents", + "Status: On", + "Environment variable: AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE", + "", + "Name: agents.long_running", + "Description: Long-running Responses invocations", + "Status: Off", + "Environment variable: AZD_FOUNDRY_EXT_ENABLE_AGENTS_LONG_RUNNING", + "", + }, "\n"), out.String()) +} + +func TestConfigListAlphaJSON(t *testing.T) { + var out bytes.Buffer + err := runConfigListAlpha(t.Context(), &out, "json", + func(_ context.Context, feature featureflags.Feature) (bool, error) { + return feature == featureflags.LongRunning, nil + }) + require.NoError(t, err) + var response alphaFeatureListResponse + require.NoError(t, json.Unmarshal(out.Bytes(), &response)) + require.Len(t, response.Features, len(featureflags.All())) + for i, feature := range featureflags.All() { + item := response.Features[i] + require.Equal(t, feature.ID, item.Name) + require.Equal(t, feature.EnvVar, item.EnvVar) + require.Equal(t, feature.Description, item.Description) + want := "Off" + if feature == featureflags.LongRunning { + want = "On" + } + require.Equal(t, want, item.Status) + require.Empty(t, item.Error) + } + require.NotContains(t, out.String(), "error") +} + +func TestConfigListAlphaReportsEachReadError(t *testing.T) { + for _, format := range []string{"text", "json"} { + t.Run(format, func(t *testing.T) { + var out bytes.Buffer + err := runConfigListAlpha(t.Context(), &out, format, + func(_ context.Context, feature featureflags.Feature) (bool, error) { + if feature == featureflags.Voice { + return false, errors.New("cannot read selected environment") + } + return true, nil + }) + require.ErrorContains(t, err, "1 of 3") + text := out.String() + for _, feature := range featureflags.All() { + require.Contains(t, text, feature.ID, "every feature is listed despite the error") + } + require.Contains(t, text, "cannot read selected environment") + if format == "text" { + require.Contains(t, text, "Name: agents.voice\n"+ + "Description: Voice, prompt-voice, and hosted voice agents\n"+ + "Status: Unknown\n"+ + "Environment variable: AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE\n"+ + "Error: cannot read selected environment") + require.Contains(t, text, "Name: agents.long_running\n"+ + "Description: Long-running Responses invocations\nStatus: On") + } + }) + } +} + +func TestConfigListAlphaCommandReadsSelectedEnvironment(t *testing.T) { + clearPreviewProcess(t) + t.Chdir(t.TempDir()) + _, env := startPreviewCommandServer(t, map[string]any{ + "all": "on", + "agents": map[string]any{"voice": "on"}, + }, &azdext.ProjectConfig{Path: t.TempDir()}, nil) + env.mu.Lock() + env.values = []*azdext.KeyValue{{Key: featureflags.LongRunning.EnvVar, Value: "true"}} + env.mu.Unlock() + + text, err := executeRoot(t, "config", "list-alpha", "-e", "selected-env", "--output", "json") + require.NoError(t, err) + var response alphaFeatureListResponse + require.NoError(t, json.Unmarshal([]byte(text), &response)) + states := map[string]string{} + for _, item := range response.Features { + states[item.Name] = item.Status + } + require.Equal(t, map[string]string{ + "agents.managed_harness": "Off", "agents.voice": "On", "agents.long_running": "On", + }, states) + env.mu.Lock() + require.Equal(t, "selected-env", env.selected) + env.mu.Unlock() + + t.Setenv(featureflags.ManagedHarness.EnvVar, "true") + text, err = executeRoot(t, "config", "list-alpha", "-e", "selected-env") + require.NoError(t, err) + require.Contains(t, text, "Name: agents.managed_harness\n"+ + "Description: Prompt agents that run on a Copilot harness\nStatus: On") + require.NotContains(t, strings.ToLower(text), "azure.yaml") + require.NotContains(t, strings.ToLower(text), "source") +} + +func TestConfigListAlphaCommandUsesProcessValue(t *testing.T) { + clearPreviewProcess(t) + t.Setenv(featureflags.Voice.EnvVar, "1") + startPreviewCommandServer(t, nil, nil, nil) + text, err := executeRoot(t, "config", "list-alpha") + require.NoError(t, err) + require.Contains(t, text, "Name: agents.voice\n"+ + "Description: Voice, prompt-voice, and hosted voice agents\nStatus: On") + require.Contains(t, text, "Name: agents.long_running\n"+ + "Description: Long-running Responses invocations\nStatus: Off") +} + +func TestConfigListAlphaRejectsUnsupportedOutput(t *testing.T) { + clearPreviewProcess(t) + _, err := executeRoot(t, "config", "list-alpha", "--output", "raw") + require.ErrorContains(t, err, "raw") +} + +func TestFeatureCommandRemoved(t *testing.T) { + clearPreviewProcess(t) + for _, child := range NewRootCommand().Commands() { + require.NotEqual(t, "feature", child.Name()) + } +} diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/json_output.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/json_output.go index 311801e48be..5dbb54cd36b 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/json_output.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/json_output.go @@ -6,13 +6,20 @@ package cmd import ( "encoding/json" "fmt" + "io" + "os" ) func emitJSON(value any) error { + return writeJSON(os.Stdout, value) +} + +// writeJSON writes value as indented JSON followed by a newline. +func writeJSON(w io.Writer, value any) error { data, err := json.MarshalIndent(value, "", " ") if err != nil { return fmt.Errorf("failed to marshal JSON output: %w", err) } - fmt.Println(string(data)) - return nil + _, err = fmt.Fprintln(w, string(data)) + return err } diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go index 78f2f665ad9..a547c65e70a 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go @@ -109,6 +109,7 @@ func NewRootCommand() *cobra.Command { rootCmd.AddCommand(newStateStoresCommand(extCtx)) rootCmd.AddCommand(newInvocationsCommand(extCtx)) rootCmd.AddCommand(newSampleCommand(extCtx)) + rootCmd.AddCommand(newConfigCommand(extCtx)) rootCmd.AddCommand(newDoctorCommand()) rootCmd.AddCommand(newCodeCommand(extCtx)) diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go index c025bc294f2..a25e78f17c3 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go @@ -25,6 +25,7 @@ func TestRootCommand_PublicPreviewCommandsVisible(t *testing.T) { for _, name := range []string{ "add", "code", + "config", "connection", "delete", "doctor", diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config.snap b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config.snap new file mode 100644 index 00000000000..9e015209256 --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config.snap @@ -0,0 +1,23 @@ +Inspect agent extension configuration. + +Use 'azd config' to change azd user configuration. + +Usage + azd ai agent config [command] + +Available Commands + list-alpha : Display the list of agent features in alpha stage. + +Global Flags + -C, --cwd string : Sets the current working directory + --debug : Enables debug and diagnostics logging + -e, --environment string : The name of the environment to use + -h, --help : help for config + --no-prompt : Runs without prompts. Uses existing values; fails if any required value or decision cannot be resolved automatically. + -o, --output string : The output format (default "default") + +Examples + Display the agent features in alpha stage + azd ai agent config list-alpha + +Use azd ai agent config [command] --help to view examples and more information about a specific command. diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config_list-alpha.snap b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config_list-alpha.snap new file mode 100644 index 00000000000..d2c8ee50e39 --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config_list-alpha.snap @@ -0,0 +1,27 @@ +Display the list of agent features in alpha stage with the status of each +in the selected environment and the environment variable that turns it on. + +Use -e/--environment to read another azd environment. When a status cannot be +read, the entry reports the error and the command exits with an error after +listing every feature. + +Usage + azd ai agent config list-alpha [flags] + +Global Flags + -C, --cwd string : Sets the current working directory + --debug : Enables debug and diagnostics logging + -e, --environment string : The name of the environment to use + -h, --help : help for list-alpha + --no-prompt : Runs without prompts. Uses existing values; fails if any required value or decision cannot be resolved automatically. + -o, --output string : The output format (supported: json, text) (default "text") + +Examples + Display the agent features in alpha stage + azd ai agent config list-alpha + + Display the features for another environment as JSON + azd ai agent config list-alpha -e dev --output json + + Turn on a specific alpha feature + azd config set alpha. on diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap index c2ae7c29ca6..68f8b4c594a 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap @@ -17,6 +17,7 @@ Usage Available Commands add : Add a file-backed Agent service to the current azd project. code : Manage agent source code. (Preview) + config : Inspect agent extension configuration. connection : Manage connection service dependencies for an agent. delete : Delete an agent. doctor : Diagnose problems with an azd ai agent project. diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelpPreviewsOff-root.snap b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelpPreviewsOff-root.snap index b7675d946d6..1c400fa52e5 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelpPreviewsOff-root.snap +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelpPreviewsOff-root.snap @@ -17,6 +17,7 @@ Usage Available Commands add : Add a file-backed Agent service to the current azd project. code : Manage agent source code. (Preview) + config : Inspect agent extension configuration. connection : Manage connection service dependencies for an agent. delete : Delete an agent. doctor : Diagnose problems with an azd ai agent project. diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md index 44eab4c48bb..dc4e22d596c 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md @@ -345,6 +345,7 @@ in any order, any time. | `tier0/0.28-agent-add-multiple-projects.yaml` | `agent add` rejects multiple Project services before mutation | | `tier0/0.29-agent-add-path-safety.yaml` | `agent add` rejects definition and source paths outside the project | | `tier0/0.30-agent-add-wrong-host-collision.yaml` | `agent add` rejects a target service key owned by a non-Agent host without mutating `azure.yaml` | +| `tier0/0.32-config-list-alpha.yaml` | `config list-alpha` text and JSON output for agent alpha features | The invocation lifecycle scenarios above are offline help/validation checks, not live execution tests. They do not require a deployed long-running agent or add Tier 2 provisioning dependencies. Actual HTTP diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml index d2f9e26b8bf..4336adf9415 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml @@ -12,7 +12,7 @@ env: goals: - "Wait for the help output to render (it includes an ASCII-art banner and a 'Usage' section)." - - "Confirm the 'Available Commands' list includes: code, delete, doctor, endpoint, eval, files, init, invocations, invoke, monitor, optimize, pack, publish, run, sample, sessions, show, version." + - "Confirm the 'Available Commands' list includes: code, config, delete, doctor, endpoint, eval, files, init, invocations, invoke, monitor, optimize, pack, publish, run, sample, sessions, show, version." - "Confirm the global flags are listed: --cwd/-C, --debug, --environment/-e, --no-prompt, --output/-o." - "Confirm the Environments & Environment Variables section explains .azure//.env, azd env new/select/set/get-values, FOUNDRY_PROJECT_ENDPOINT, AGENT_ deployment values, and the service-level env mapping for runtime settings." - "Compare root, init, and files list help: headings should be consistently bold and underlined, command examples and flags highlighted, and usage paths start with azd ai agent. With NO_COLOR=1, the same content must remain readable without escape sequences." diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml new file mode 100644 index 00000000000..c5a88bd0e7d --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml @@ -0,0 +1,16 @@ +# Tier 0: read-only alpha feature listing; no auth or Azure cost. +name: "config-list-alpha" +command: "azd ai agent config list-alpha" +cwd: "/tmp" +tags: ["tier:0", "cmd:config", "parallel-safe"] +goals: + - "Confirm the command exits with code 0 without a picker, login prompt, or project mutation." + - "Confirm the output lists agents.managed_harness, agents.voice, and agents.long_running as blocks with Name, Description, Status: Off, and an AZD_FOUNDRY_EXT_ENABLE_AGENTS_* Environment variable line, in the layout of 'azd config list-alpha'." + - "Run 'AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE=true azd ai agent config list-alpha'. Confirm only agents.voice shows Status: On." + - "Run 'azd ai agent config list-alpha --output json'. Confirm the output is a JSON object with a features array whose items have name, description, status, and envVar fields." + - "Run 'azd ai agent config list-alpha --output raw'. Confirm it exits non-zero and names the supported formats." + - "Run 'azd ai agent config --help' and 'azd ai agent config list-alpha --help'. Confirm both exit with code 0." + - "Run 'azd ai agent feature list'. Confirm it does not list features." + - "Confirm the output shows no field or text that names where a feature was turned on." + - "Take a screenshot of the text and JSON output." + - "Report a finding if a registered feature is missing, a status is wrong, or the command prompts or mutates a project." From 9c9e244c18a3ad25d97f29e3aa445331604c0a3b Mon Sep 17 00:00:00 2001 From: Hui Miao Date: Thu, 22 Oct 2026 08:20:26 +0800 Subject: [PATCH 02/10] fix: keep read errors from config list-alpha and assert JSON error items --- .../azure.ai.agents/internal/cmd/config.go | 16 ++++++--- .../internal/cmd/config_test.go | 36 +++++++++++++++++++ 2 files changed, 48 insertions(+), 4 deletions(-) diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go index dd5b6b56faf..9fadb15b0ed 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go @@ -5,6 +5,7 @@ package cmd import ( "context" + "errors" "fmt" "io" "strings" @@ -106,7 +107,7 @@ listing every feature.`, func runConfigListAlpha(ctx context.Context, out io.Writer, format string, read alphaFeatureReader) error { features := featureflags.All() items := make([]AlphaFeatureItem, 0, len(features)) - failed := 0 + var failures []error for _, feature := range features { item := AlphaFeatureItem{ Name: feature.ID, @@ -119,7 +120,11 @@ func runConfigListAlpha(ctx context.Context, out io.Writer, format string, read case err != nil: item.Status = alphaStatusUnknown item.Error = err.Error() - failed++ + if errors.Is(err, context.Canceled) { + failures = append([]error{err}, failures...) + } else { + failures = append(failures, err) + } case enabled: item.Status = alphaStatusOn } @@ -135,8 +140,11 @@ func runConfigListAlpha(ctx context.Context, out io.Writer, format string, read if err != nil { return err } - if failed > 0 { - return fmt.Errorf("could not read the status of %d of %d alpha features", failed, len(items)) + if len(failures) > 0 { + // Keep the read errors so cancellation and structured errors keep + // their classification. Cancellation is listed first. + return fmt.Errorf("could not read the status of %d of %d alpha features: %w", + len(failures), len(items), errors.Join(failures...)) } return nil } diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go index 1fa78b8aaeb..21c4da2dcb0 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go @@ -8,6 +8,7 @@ import ( "context" "encoding/json" "errors" + "io" "strings" "testing" @@ -15,6 +16,8 @@ import ( "github.com/azure/azure-dev/cli/azd/pkg/azdext" "github.com/stretchr/testify/require" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" ) func TestConfigListAlphaText(t *testing.T) { @@ -85,6 +88,19 @@ func TestConfigListAlphaReportsEachReadError(t *testing.T) { require.Contains(t, text, feature.ID, "every feature is listed despite the error") } require.Contains(t, text, "cannot read selected environment") + if format == "json" { + var response alphaFeatureListResponse + require.NoError(t, json.Unmarshal(out.Bytes(), &response)) + for _, item := range response.Features { + if item.Name == featureflags.Voice.ID { + require.Equal(t, "Unknown", item.Status) + require.Equal(t, "cannot read selected environment", item.Error) + } else { + require.Equal(t, "On", item.Status) + require.Empty(t, item.Error) + } + } + } if format == "text" { require.Contains(t, text, "Name: agents.voice\n"+ "Description: Voice, prompt-voice, and hosted voice agents\n"+ @@ -98,6 +114,26 @@ func TestConfigListAlphaReportsEachReadError(t *testing.T) { } } +func TestConfigListAlphaKeepsReadErrorChain(t *testing.T) { + notFound := status.Error(codes.NotFound, "environment not found") + err := runConfigListAlpha(t.Context(), io.Discard, "text", + func(_ context.Context, feature featureflags.Feature) (bool, error) { + switch feature { + case featureflags.Voice: + return false, notFound + case featureflags.LongRunning: + return false, context.Canceled + } + return false, nil + }) + require.ErrorContains(t, err, "2 of 3") + require.ErrorIs(t, err, context.Canceled) + require.ErrorIs(t, err, notFound) + joined, ok := errors.Unwrap(err).(interface{ Unwrap() []error }) + require.True(t, ok) + require.ErrorIs(t, joined.Unwrap()[0], context.Canceled, "cancellation takes precedence") +} + func TestConfigListAlphaCommandReadsSelectedEnvironment(t *testing.T) { clearPreviewProcess(t) t.Chdir(t.TempDir()) From 0e3453c4bbdf217235e1ce4a7973b505305ecc30 Mon Sep 17 00:00:00 2001 From: Hui Miao Date: Thu, 22 Oct 2026 08:20:26 +0800 Subject: [PATCH 03/10] test: document cmd:config tag and tag config help coverage --- .../tests/cli-interactive-tester-scenarios/README.md | 2 +- .../tier0/0.32-config-list-alpha.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md index dc4e22d596c..84a799e800b 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md @@ -520,7 +520,7 @@ grouping — colons are treated as ordinary characters by the filter): | Namespace | Values | Meaning | |---|---|---| | `tier:N` | `tier:0`, `tier:1`, `tier:1b`, `tier:2` | The tier the scenario belongs to (same axis as the directory's four sections above). Use this to express cost / auth profile in one tag. | -| `cmd:*` | `cmd:init`, `cmd:show`, `cmd:invoke`, `cmd:invocations`, `cmd:sessions`, `cmd:files`, `cmd:state-stores`, `cmd:monitor`, `cmd:endpoint`, `cmd:run`, `cmd:doctor`, `cmd:eval`, `cmd:optimize`, `cmd:sample`, `cmd:down`, `cmd:provision`, `cmd:deploy`, `cmd:version`, `cmd:help`, `cmd:code`, `cmd:delete`, `cmd:toolbox`, `cmd:connection`, `cmd:add` | The top-level `azd ai agent` (or `azd`) command(s) the scenario exercises. Multi-command scenarios (e.g. `2.12-run-local-and-invoke-local` runs both `run` and `invoke --local`; `2.00-setup` runs `init` + `provision` + `deploy`) carry multiple `cmd:*` tags. `cmd:add` tags the Agent add command; `cmd:toolbox` and `cmd:connection` cover Agent dependency composition, not the sibling extensions' resource lifecycle commands. | +| `cmd:*` | `cmd:init`, `cmd:show`, `cmd:invoke`, `cmd:invocations`, `cmd:sessions`, `cmd:files`, `cmd:state-stores`, `cmd:monitor`, `cmd:endpoint`, `cmd:run`, `cmd:doctor`, `cmd:eval`, `cmd:optimize`, `cmd:sample`, `cmd:down`, `cmd:provision`, `cmd:deploy`, `cmd:version`, `cmd:help`, `cmd:code`, `cmd:delete`, `cmd:toolbox`, `cmd:connection`, `cmd:add`, `cmd:config` | The top-level `azd ai agent` (or `azd`) command(s) the scenario exercises. Multi-command scenarios (e.g. `2.12-run-local-and-invoke-local` runs both `run` and `invoke --local`; `2.00-setup` runs `init` + `provision` + `deploy`) carry multiple `cmd:*` tags. `cmd:add` tags the Agent add command; `cmd:toolbox` and `cmd:connection` cover Agent dependency composition, not the sibling extensions' resource lifecycle commands. | | traits | `parallel-safe`, `serial-only`, `negative-path`, `picker`, `verify-deploy`, `manual-seed` | `parallel-safe` ↔ `serial-only` are mutually exclusive: all Tier 0 / Tier 1 / Tier 1b scenarios are `parallel-safe`, all Tier 2 are `serial-only`. `negative-path` flags arg-/CLI-validation scenarios that assert errors or non-zero exit codes rather than happy-path success. `picker` flags scenarios whose primary purpose is exercising interactive picker UX. `verify-deploy` flags Tier 1b scenarios that verify a Tier 1 scaffold deploys. `manual-seed` flags an opt-in scenario that needs an externally prepared fixture before it runs; filtering by this tag does not create the fixture. | The tool returns matching `manual-seed` rows, but runner plans exclude them diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml index c5a88bd0e7d..23e1d9cfef7 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml @@ -2,7 +2,7 @@ name: "config-list-alpha" command: "azd ai agent config list-alpha" cwd: "/tmp" -tags: ["tier:0", "cmd:config", "parallel-safe"] +tags: ["tier:0", "cmd:config", "cmd:help", "parallel-safe"] goals: - "Confirm the command exits with code 0 without a picker, login prompt, or project mutation." - "Confirm the output lists agents.managed_harness, agents.voice, and agents.long_running as blocks with Name, Description, Status: Off, and an AZD_FOUNDRY_EXT_ENABLE_AGENTS_* Environment variable line, in the layout of 'azd config list-alpha'." From 2da7a7a696f6d34eef6178c7ac2f01bdb89e7173 Mon Sep 17 00:00:00 2001 From: Hui Miao Date: Thu, 22 Oct 2026 08:24:38 +0800 Subject: [PATCH 04/10] fix: keep list-alpha error message short while unwrapping read errors --- .../azure.ai.agents/internal/cmd/config.go | 19 +++++++++++++++---- .../internal/cmd/config_test.go | 5 +++-- 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go index 9fadb15b0ed..95addecce66 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go @@ -141,14 +141,25 @@ func runConfigListAlpha(ctx context.Context, out io.Writer, format string, read return err } if len(failures) > 0 { - // Keep the read errors so cancellation and structured errors keep - // their classification. Cancellation is listed first. - return fmt.Errorf("could not read the status of %d of %d alpha features: %w", - len(failures), len(items), errors.Join(failures...)) + return &alphaReadError{total: len(items), causes: failures} } return nil } +// alphaReadError reports how many statuses could not be read. The listing +// already shows each message, so Error stays short while Unwrap keeps the +// read errors for classification. Cancellation is listed first. +type alphaReadError struct { + total int + causes []error +} + +func (e *alphaReadError) Error() string { + return fmt.Sprintf("could not read the status of %d of %d alpha features", len(e.causes), e.total) +} + +func (e *alphaReadError) Unwrap() []error { return e.causes } + // writeAlphaFeatures writes one block per feature in the layout of // `azd config list-alpha`, plus the environment variable. func writeAlphaFeatures(out io.Writer, items []AlphaFeatureItem) error { diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go index 21c4da2dcb0..8ed832e9f63 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go @@ -129,9 +129,10 @@ func TestConfigListAlphaKeepsReadErrorChain(t *testing.T) { require.ErrorContains(t, err, "2 of 3") require.ErrorIs(t, err, context.Canceled) require.ErrorIs(t, err, notFound) - joined, ok := errors.Unwrap(err).(interface{ Unwrap() []error }) + require.Equal(t, "could not read the status of 2 of 3 alpha features", err.Error()) + readErr, ok := errors.AsType[*alphaReadError](err) require.True(t, ok) - require.ErrorIs(t, joined.Unwrap()[0], context.Canceled, "cancellation takes precedence") + require.ErrorIs(t, readErr.Unwrap()[0], context.Canceled, "cancellation takes precedence") } func TestConfigListAlphaCommandReadsSelectedEnvironment(t *testing.T) { From 2eb8170163b2b7dc52ed3d8695b5a1c141cbde39 Mon Sep 17 00:00:00 2001 From: Hui Miao Date: Thu, 22 Oct 2026 09:34:38 +0800 Subject: [PATCH 05/10] test: run the list-alpha scenario with an isolated config copy Copy the real azd config dir, including installed extensions, into a per instance dir, drop every alpha key, and clear the feature variables so the Off baseline does not depend on the caller's opt-ins. A post hook removes the copy. --- .../README.md | 7 ++- .../tier0/0.32-config-list-alpha.yaml | 62 ++++++++++++++++++- 2 files changed, 64 insertions(+), 5 deletions(-) diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md index 84a799e800b..2e2166614c8 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md @@ -345,7 +345,7 @@ in any order, any time. | `tier0/0.28-agent-add-multiple-projects.yaml` | `agent add` rejects multiple Project services before mutation | | `tier0/0.29-agent-add-path-safety.yaml` | `agent add` rejects definition and source paths outside the project | | `tier0/0.30-agent-add-wrong-host-collision.yaml` | `agent add` rejects a target service key owned by a non-Agent host without mutating `azure.yaml` | -| `tier0/0.32-config-list-alpha.yaml` | `config list-alpha` text and JSON output for agent alpha features | +| `tier0/0.32-config-list-alpha.yaml` | `config list-alpha` text and JSON output with an isolated config copy and no inherited alpha opt-ins | The invocation lifecycle scenarios above are offline help/validation checks, not live execution tests. They do not require a deployed long-running agent or add Tier 2 provisioning dependencies. Actual HTTP @@ -691,6 +691,8 @@ How they're used here: (`1.04-init-from-code`, `1.06-init-deploy-mode-code`) also copy a committed Python fixture into the dir so the source exists before the wizard's "Use the code in the current directory" flow inspects it (see [Fixtures](#fixtures)). +- `0.32` uses a `pre` config copy hook that preserves installed extensions and strips alpha keys. + A `post` hook removes the dedicated working directory. - **`pre` gh-auth guard** — the remote azure.yaml project scenarios (`1.03-init-from-azure-yaml-url`, `1.05-init-flag-agent-name`) run `gh auth status` and fail fast if GitHub CLI isn't authenticated, because downloading the project file and its sibling @@ -801,7 +803,8 @@ be run back to back in any order within a tier: - Tier 0/1 stateful scenarios **pre-wipe** their own `cwd`. Cleanup is pre-wipe **only** (no `post` delete), so the generated scaffold stays on disk for - inspection after a run while the next run still starts clean. + inspection after a run while the next run still starts clean. Scenario `0.32` also removes + its config copy and working directory with a `post` hook. - Tier 1b scenarios use the prerequisite worker's verified `scaffold_dir` and preserve that Tier 1 scaffold, but use an always-run `post` hook to down their isolated Azure resources. Cleanup failure is a separate scenario failure and leaves the local environment available diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml index 23e1d9cfef7..090529b9299 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml @@ -1,9 +1,62 @@ -# Tier 0: read-only alpha feature listing; no auth or Azure cost. +# Tier 0: read-only alpha feature listing with a dedicated config copy, no auth or Azure cost. name: "config-list-alpha" -command: "azd ai agent config list-alpha" -cwd: "/tmp" +command: 'AZD_CONFIG_DIR="$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" azd ai agent config list-alpha' +cwd: "~/working/azd-agents-t0-config-list-alpha-{instance}" tags: ["tier:0", "cmd:config", "cmd:help", "parallel-safe"] + +env: + AZD_FOUNDRY_EXT_ENABLE_AGENTS_MANAGED_HARNESS: "" + AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE: "" + AZD_FOUNDRY_EXT_ENABLE_AGENTS_LONG_RUNNING: "" + AZURE_ENV_NAME: "" + AZURE_DEV_COLLECT_TELEMETRY: "no" + AZD_SKIP_UPDATE_CHECK: "true" + NO_COLOR: "1" + +pre: + - run: >- + rm -rf -- "$HOME/working/azd-agents-t0-config-list-alpha-{instance}" && + mkdir -p -- "$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" + cwd: "~/working" + name: "reset the dedicated offline directory" + - run: | + set -e + if [ -n "$AZD_CONFIG_DIR" ]; then + source_config="$AZD_CONFIG_DIR" + else + source_config="$HOME/.azd" + fi + cp -a -- "$source_config/." "$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config/" + python3 - "$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config/config.json" <<'PY' + import json + import pathlib + import sys + + path = pathlib.Path(sys.argv[1]) + if path.exists(): + with path.open() as stream: + data = json.load(stream) + data.pop("alpha", None) + with path.open("w") as stream: + json.dump(data, stream, indent=2) + stream.write("\n") + with path.open() as stream: + json.load(stream) + PY + cwd: "~/working" + name: "copy the real config and extensions and remove all alpha opt-ins" + +post: + - run: 'rm -rf -- "$HOME/working/azd-agents-t0-config-list-alpha-{instance}"' + cwd: "~/working" + name: "remove the dedicated config copy and working directory" + goals: + - >- + Run every listed product command with the same + AZD_CONFIG_DIR="$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" + prefix, including the voice process-variable check and help commands. + Keep this working directory free of any project. - "Confirm the command exits with code 0 without a picker, login prompt, or project mutation." - "Confirm the output lists agents.managed_harness, agents.voice, and agents.long_running as blocks with Name, Description, Status: Off, and an AZD_FOUNDRY_EXT_ENABLE_AGENTS_* Environment variable line, in the layout of 'azd config list-alpha'." - "Run 'AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE=true azd ai agent config list-alpha'. Confirm only agents.voice shows Status: On." @@ -12,5 +65,8 @@ goals: - "Run 'azd ai agent config --help' and 'azd ai agent config list-alpha --help'. Confirm both exit with code 0." - "Run 'azd ai agent feature list'. Confirm it does not list features." - "Confirm the output shows no field or text that names where a feature was turned on." + - >- + Confirm the real config directory is unchanged, using the caller's nonempty AZD_CONFIG_DIR + or otherwise $HOME/.azd. No alpha key may be added or removed in its config.json. - "Take a screenshot of the text and JSON output." - "Report a finding if a registered feature is missing, a status is wrong, or the command prompts or mutates a project." From 7cf859518e1617f076ca1dddaa9bda4b547fc4d6 Mon Sep 17 00:00:00 2001 From: Hui Miao Date: Thu, 22 Oct 2026 09:54:56 +0800 Subject: [PATCH 06/10] fix: reject an explicit empty environment in config list-alpha Return the --environment validation error before connecting to azd so list-alpha never reads the default environment for an empty selection. --- cli/azd/extensions/azure.ai.agents/README.md | 6 ++++-- .../docs/specs/preview-feature-flags.md | 2 ++ .../azure.ai.agents/internal/cmd/config.go | 3 +++ .../internal/cmd/config_test.go | 20 +++++++++++++++++++ 4 files changed, 29 insertions(+), 2 deletions(-) diff --git a/cli/azd/extensions/azure.ai.agents/README.md b/cli/azd/extensions/azure.ai.agents/README.md index 3f246754ccb..724c6e3ef0f 100644 --- a/cli/azd/extensions/azure.ai.agents/README.md +++ b/cli/azd/extensions/azure.ai.agents/README.md @@ -36,8 +36,10 @@ do not turn them on. Ordinary hosted and plain prompt agents need no opt-in. `azd ai agent config list-alpha` lists each feature with its name, description, status in the selected environment, and environment variable, in the layout of -`azd config list-alpha`. Use `-e` to read another environment and -`--output json` for scripts. +`azd config list-alpha`. + +- `-e/--environment` selects another environment. An explicit empty environment is rejected. +- `--output json` sets the output format for scripts. ```bash azd ai agent config list-alpha diff --git a/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md b/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md index eb3752d89d9..7592342d26e 100644 --- a/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md +++ b/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md @@ -153,6 +153,8 @@ The [README](../../README.md#preview-feature-opt-ins) lists the equivalent comma `azd ai agent config list-alpha` mirrors `azd config list-alpha`. It prints every feature from `All` as a block with `Name`, `Description`, `Status` (`On` or `Off`), and `Environment variable`. It calls `IsEnabled` once per feature with the environment selected by `-e`, so the status matches what enforcement sees. It prints no source for an opt-in because the API returns only a decision. +An explicit empty environment is rejected before any source is read. + The default text output prints `Status: Unknown` and an `Error` line for a feature whose status cannot be read. `--output json` prints `{"features":[...]}` with `name`, `description`, `status`, `envVar`, and an `error` field when the read failed. Both formats list every feature before the command exits with an error that counts the failed reads. The command is read only and needs no authentication. ## Documented enablement methods diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go index 95addecce66..d861d61bb04 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go @@ -82,6 +82,9 @@ listing every feature.`, Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { ctx := cmd.Context() + if err := featureflags.CheckEnvironmentFlag(cmd.Flags()); err != nil { + return err + } client, err := azdext.NewAzdClient() if err != nil { return fmt.Errorf("connecting to azd to read alpha features: %w", err) diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go index 8ed832e9f63..83c602fd65f 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go @@ -12,6 +12,7 @@ import ( "strings" "testing" + "azureaiagent/internal/exterrors" "azureaiagent/internal/featureflags" "github.com/azure/azure-dev/cli/azd/pkg/azdext" @@ -188,6 +189,25 @@ func TestConfigListAlphaRejectsUnsupportedOutput(t *testing.T) { require.ErrorContains(t, err, "raw") } +func TestConfigListAlphaRejectsEmptyEnvironment(t *testing.T) { + for _, value := range []string{"", " \t "} { + for _, flag := range []string{"--environment", "-e"} { + t.Run(flag+"/"+value, func(t *testing.T) { + clearPreviewProcess(t) + t.Setenv("AZD_ENVIRONMENT", "default-env") + + text, err := executeRoot(t, "config", "list-alpha", flag, value) + localErr, ok := errors.AsType[*azdext.LocalError](err) + require.True(t, ok, "%v", err) + require.Equal(t, exterrors.CodeInvalidParameter, localErr.Code) + require.Equal(t, azdext.LocalErrorCategoryValidation, localErr.Category) + require.Contains(t, localErr.Message, "--environment") + require.Empty(t, text) + }) + } + } +} + func TestFeatureCommandRemoved(t *testing.T) { clearPreviewProcess(t) for _, child := range NewRootCommand().Commands() { From e435a33c26b27e296c5cd3db934814c075e7081c Mon Sep 17 00:00:00 2001 From: Hui Miao Date: Thu, 22 Oct 2026 10:19:13 +0800 Subject: [PATCH 07/10] fix: reject an empty environment before the list-alpha telemetry read --- .../docs/specs/preview-feature-flags.md | 4 ++-- .../azure.ai.agents/internal/cmd/config.go | 6 ++---- .../internal/cmd/config_test.go | 14 +++++++++++++- .../azure.ai.agents/internal/cmd/root.go | 7 +++++++ .../azure.ai.agents/internal/cmd/root_test.go | 19 +++++++++++++++++++ 5 files changed, 43 insertions(+), 7 deletions(-) diff --git a/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md b/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md index 7592342d26e..f779006fc40 100644 --- a/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md +++ b/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md @@ -100,7 +100,7 @@ enabled, err := featureflags.IsEnabled( Enforcement has two layers. Each layer has one check. -**Command layer.** Commands and flags carry annotations that name a registered feature. The root `PersistentPreRunE` runs after the SDK pre-run, so `-e` and `AZD_ENVIRONMENT` are already applied. It calls `enforcePreviewFeatures` for the executing command with its args. Whenever the gate would reject, it first runs the command's local argument checks from `validateBeforePreview` so local conflicts are reported before `feature_not_enabled` or an empty environment error. These checks use only flags and args, with no source reads or state changes. The existing debug logging and telemetry steps keep their order. The gate makes no host reads for a command without a preview annotation or a set preview flag. A bound environment error only fails commands that perform a preview check. +**Command layer.** Commands and flags carry annotations that name a registered feature. The root `PersistentPreRunE` runs after the SDK pre-run, so `-e` and `AZD_ENVIRONMENT` are already applied. It calls `enforcePreviewFeatures` for the executing command with its args. Whenever the gate would reject, it first runs the command's local argument checks from `validateBeforePreview` so local conflicts are reported before `feature_not_enabled` or an empty environment error. These checks use only flags and args, with no source reads or state changes. The existing debug logging and telemetry steps keep their order. The gate makes no host reads for a command without a preview annotation or a set preview flag. A bound environment error only fails commands that perform a preview check or carry `rejectEmptyEnvironmentAnnotation`. | Annotation | Placed on | Effect | | --- | --- | --- | @@ -153,7 +153,7 @@ The [README](../../README.md#preview-feature-opt-ins) lists the equivalent comma `azd ai agent config list-alpha` mirrors `azd config list-alpha`. It prints every feature from `All` as a block with `Name`, `Description`, `Status` (`On` or `Off`), and `Environment variable`. It calls `IsEnabled` once per feature with the environment selected by `-e`, so the status matches what enforcement sees. It prints no source for an opt-in because the API returns only a decision. -An explicit empty environment is rejected before any source is read. +An explicit empty environment is rejected before any source is read. The root pre-run rejects it before the project read for telemetry. The default text output prints `Status: Unknown` and an `Error` line for a feature whose status cannot be read. `--output json` prints `{"features":[...]}` with `name`, `description`, `status`, `envVar`, and an `error` field when the read failed. Both formats list every feature before the command exits with an error that counts the failed reads. The command is read only and needs no authentication. diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go index d861d61bb04..97dc7295d00 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go @@ -79,12 +79,10 @@ listing every feature.`, # Turn on a specific alpha feature azd config set alpha. on`, - Args: cobra.NoArgs, + Annotations: map[string]string{rejectEmptyEnvironmentAnnotation: "true"}, + Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { ctx := cmd.Context() - if err := featureflags.CheckEnvironmentFlag(cmd.Flags()); err != nil { - return err - } client, err := azdext.NewAzdClient() if err != nil { return fmt.Errorf("connecting to azd to read alpha features: %w", err) diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go index 83c602fd65f..99293da0bb9 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go @@ -195,19 +195,31 @@ func TestConfigListAlphaRejectsEmptyEnvironment(t *testing.T) { t.Run(flag+"/"+value, func(t *testing.T) { clearPreviewProcess(t) t.Setenv("AZD_ENVIRONMENT", "default-env") + _, env := startPreviewCommandServer(t, nil, nil, nil) text, err := executeRoot(t, "config", "list-alpha", flag, value) localErr, ok := errors.AsType[*azdext.LocalError](err) require.True(t, ok, "%v", err) require.Equal(t, exterrors.CodeInvalidParameter, localErr.Code) require.Equal(t, azdext.LocalErrorCategoryValidation, localErr.Category) - require.Contains(t, localErr.Message, "--environment") + require.Equal(t, "the --environment flag was set to an empty value", localErr.Message) require.Empty(t, text) + require.Zero(t, env.reads.Load(), "no project read or telemetry report should reach the host") }) } } } +func TestConfigListAlphaHelpAcceptsEmptyEnvironment(t *testing.T) { + clearPreviewProcess(t) + _, env := startPreviewCommandServer(t, nil, nil, nil) + + text, err := executeRoot(t, "config", "list-alpha", "-e", "", "--help") + require.NoError(t, err) + require.Contains(t, text, "Display the list of agent features in alpha stage") + require.Zero(t, env.reads.Load()) +} + func TestFeatureCommandRemoved(t *testing.T) { clearPreviewProcess(t) for _, child := range NewRootCommand().Commands() { diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go index 2e905488c3e..3b92e447813 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go @@ -14,6 +14,8 @@ import ( "github.com/spf13/cobra" ) +const rejectEmptyEnvironmentAnnotation = "azure.ai.agents/reject-empty-environment" + func NewRootCommand() *cobra.Command { rootCmd, extCtx := azdext.NewExtensionRootCommand(azdext.ExtensionCommandOptions{ Name: "agent", @@ -52,6 +54,11 @@ func NewRootCommand() *cobra.Command { if err := enforcePreviewFeatures(cmd, args, extCtx.Environment); err != nil { return err } + if cmd.Annotations[rejectEmptyEnvironmentAnnotation] == "true" { + if err := featureflags.EnvironmentError(cmd.Context()); err != nil { + return err + } + } operation := telemetryOperation(cmd.CommandPath()) switch operation { case "deploy", "init", "listen", "metadata", "version": diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go index a25e78f17c3..d6c28318915 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go @@ -9,9 +9,28 @@ import ( "strings" "testing" + "github.com/spf13/cobra" "github.com/stretchr/testify/require" ) +func TestRootCommandInvokeAcceptsEmptyEnvironment(t *testing.T) { + clearPreviewProcess(t) + _, env := startPreviewCommandServer(t, nil, nil, nil) + root := NewRootCommand() + invoke, _, err := root.Find([]string{"invoke"}) + require.NoError(t, err) + ran := false + invoke.RunE = func(*cobra.Command, []string) error { + ran = true + return nil + } + root.SetArgs([]string{"invoke", "-e", "", "hello"}) + + require.NoError(t, root.ExecuteContext(t.Context())) + require.True(t, ran) + require.Positive(t, env.reads.Load(), "project telemetry should still read the project") +} + func TestRootCommand_PublicPreviewCommandsVisible(t *testing.T) { cmd := NewRootCommand() From d1bce6cfa53cdc66964a381e96b7d72720ca6faa Mon Sep 17 00:00:00 2001 From: huimiu Date: Thu, 22 Oct 2026 11:55:37 +0800 Subject: [PATCH 08/10] fix: list unknown alpha features when client setup fails --- .../azure.ai.agents/internal/cmd/config.go | 8 +- .../internal/cmd/config_test.go | 92 +++++++++++++++++++ 2 files changed, 98 insertions(+), 2 deletions(-) diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go index 97dc7295d00..cf17dda3c7f 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go @@ -85,11 +85,15 @@ listing every feature.`, ctx := cmd.Context() client, err := azdext.NewAzdClient() if err != nil { - return fmt.Errorf("connecting to azd to read alpha features: %w", err) + err = fmt.Errorf("connecting to azd to read alpha features: %w", err) + } else { + defer client.Close() } - defer client.Close() environmentName := featureflags.Environment(ctx) read := func(ctx context.Context, feature featureflags.Feature) (bool, error) { + if err != nil { + return false, err + } return featureflags.IsEnabled(ctx, client, feature, environmentName) } return runConfigListAlpha(ctx, cmd.OutOrStdout(), extCtx.OutputFormat, read) diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go index 99293da0bb9..281080c3752 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go @@ -183,6 +183,98 @@ func TestConfigListAlphaCommandUsesProcessValue(t *testing.T) { "Description: Long-running Responses invocations\nStatus: Off") } +func TestConfigListAlphaCommandReportsConstructionError(t *testing.T) { + for _, format := range []string{"text", "json"} { + t.Run(format, func(t *testing.T) { + clearPreviewProcess(t) + t.Setenv("AZD_SERVER", "dns:///%zz") + client, cause := azdext.NewAzdClient() + if client != nil { + t.Cleanup(client.Close) + } + require.Error(t, cause, "the real constructor must reject the target") + require.Nil(t, client) + + text, err := executeRoot(t, "config", "list-alpha", "--output", format) + require.Error(t, err) + readErr, ok := errors.AsType[*alphaReadError](err) + require.True(t, ok, "%v", err) + features := featureflags.All() + require.Len(t, readErr.Unwrap(), len(features)) + wrapped := readErr.Unwrap()[0] + original := errors.Unwrap(wrapped) + require.Error(t, original) + require.EqualError(t, original, cause.Error()) + require.EqualError(t, wrapped, "connecting to azd to read alpha features: "+cause.Error()) + require.ErrorIs(t, err, original) + for _, failure := range readErr.Unwrap() { + require.ErrorIs(t, failure, original) + } + if format == "json" { + var response alphaFeatureListResponse + require.NoError(t, json.Unmarshal([]byte(text), &response)) + require.Len(t, response.Features, len(features)) + for i, feature := range features { + item := response.Features[i] + require.Equal(t, feature.ID, item.Name) + require.Equal(t, feature.Description, item.Description) + require.Equal(t, feature.EnvVar, item.EnvVar) + require.Equal(t, alphaStatusUnknown, item.Status) + require.Equal(t, wrapped.Error(), item.Error) + } + } else { + require.Equal(t, len(features), strings.Count(text, "Status: Unknown\n")) + require.Equal(t, len(features), strings.Count(text, "Error: "+wrapped.Error())) + for _, feature := range features { + require.Contains(t, text, "Name: "+feature.ID+"\n") + require.Contains(t, text, "Environment variable: "+feature.EnvVar+"\n") + } + } + }) + } +} + +type alphaFailingWriter struct { + err error +} + +func (w alphaFailingWriter) Write([]byte) (int, error) { + return 0, w.err +} + +func TestConfigListAlphaCommandConstructionErrorPreservesWriteError(t *testing.T) { + for _, format := range []string{"text", "json"} { + t.Run(format, func(t *testing.T) { + clearPreviewProcess(t) + t.Setenv("AZD_SERVER", "dns:///%zz") + writeErr := errors.New("cannot write alpha features") + root := NewRootCommand() + root.SetOut(alphaFailingWriter{err: writeErr}) + root.SetErr(io.Discard) + root.SetArgs([]string{"config", "list-alpha", "--output", format}) + require.ErrorIs(t, root.Execute(), writeErr) + }) + } +} + +func TestConfigListAlphaCommandValidatesBeforeConstruction(t *testing.T) { + for _, args := range [][]string{ + {"--output", "raw"}, + {"--environment", ""}, + {"--environment", " \t "}, + } { + t.Run(strings.Join(args, "/"), func(t *testing.T) { + clearPreviewProcess(t) + t.Setenv("AZD_SERVER", "dns:///%zz") + text, err := executeRoot(t, append([]string{"config", "list-alpha"}, args...)...) + require.Error(t, err) + require.NotContains(t, err.Error(), "connecting to azd") + require.NotContains(t, err.Error(), "could not read the status") + require.Empty(t, text) + }) + } +} + func TestConfigListAlphaRejectsUnsupportedOutput(t *testing.T) { clearPreviewProcess(t) _, err := executeRoot(t, "config", "list-alpha", "--output", "raw") From f985a953924c6ccc730cd9190a5f78a9f3a0c673 Mon Sep 17 00:00:00 2001 From: huimiu Date: Thu, 22 Oct 2026 14:33:02 +0800 Subject: [PATCH 09/10] fix: isolate offline listing without copying auth state --- .../README.md | 8 ++-- .../tier0/0.32-config-list-alpha.yaml | 46 +++++++++++++------ 2 files changed, 36 insertions(+), 18 deletions(-) diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md index 2e2166614c8..7d05a21dd02 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md @@ -345,7 +345,7 @@ in any order, any time. | `tier0/0.28-agent-add-multiple-projects.yaml` | `agent add` rejects multiple Project services before mutation | | `tier0/0.29-agent-add-path-safety.yaml` | `agent add` rejects definition and source paths outside the project | | `tier0/0.30-agent-add-wrong-host-collision.yaml` | `agent add` rejects a target service key owned by a non-Agent host without mutating `azure.yaml` | -| `tier0/0.32-config-list-alpha.yaml` | `config list-alpha` text and JSON output with an isolated config copy and no inherited alpha opt-ins | +| `tier0/0.32-config-list-alpha.yaml` | `config list-alpha` text and JSON output with isolated agent registration/artifacts and no inherited alpha opt-ins | The invocation lifecycle scenarios above are offline help/validation checks, not live execution tests. They do not require a deployed long-running agent or add Tier 2 provisioning dependencies. Actual HTTP @@ -691,8 +691,10 @@ How they're used here: (`1.04-init-from-code`, `1.06-init-deploy-mode-code`) also copy a committed Python fixture into the dir so the source exists before the wizard's "Use the code in the current directory" flow inspects it (see [Fixtures](#fixtures)). -- `0.32` uses a `pre` config copy hook that preserves installed extensions and strips alpha keys. - A `post` hook removes the dedicated working directory. +- `0.32` creates a minimal config containing only the installed `azure.ai.agents` registration + and copies that extension's package, including supporting files. It does not copy auth state, + other extensions, alpha settings, or unrelated config. Missing registrations/artifacts and + symbolic links fail setup explicitly. A `post` hook removes the dedicated working directory. - **`pre` gh-auth guard** — the remote azure.yaml project scenarios (`1.03-init-from-azure-yaml-url`, `1.05-init-flag-agent-name`) run `gh auth status` and fail fast if GitHub CLI isn't authenticated, because downloading the project file and its sibling diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml index 090529b9299..5a0eb4998eb 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml @@ -1,4 +1,5 @@ -# Tier 0: read-only alpha feature listing with a dedicated config copy, no auth or Azure cost. +# Tier 0: alpha feature listing with isolated extension data, no auth or Azure cost. +# cspell:ignore shutil rglob copytree name: "config-list-alpha" command: 'AZD_CONFIG_DIR="$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" azd ai agent config list-alpha' cwd: "~/working/azd-agents-t0-config-list-alpha-{instance}" @@ -26,25 +27,40 @@ pre: else source_config="$HOME/.azd" fi - cp -a -- "$source_config/." "$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config/" - python3 - "$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config/config.json" <<'PY' + python3 - "$source_config" "$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" <<'PY' import json import pathlib + import shutil import sys - path = pathlib.Path(sys.argv[1]) - if path.exists(): - with path.open() as stream: - data = json.load(stream) - data.pop("alpha", None) - with path.open("w") as stream: - json.dump(data, stream, indent=2) - stream.write("\n") - with path.open() as stream: - json.load(stream) + source = pathlib.Path(sys.argv[1]) + target = pathlib.Path(sys.argv[2]) + with (source / "config.json").open() as stream: + data = json.load(stream) + extension_id = "azure.ai.agents" + installed = data.get("extension", {}).get("installed", {}) + if extension_id not in installed: + raise ValueError("install azure.ai.agents before running this scenario") + registration = installed[extension_id] + entry = pathlib.Path(registration["path"]) + package = pathlib.Path("extensions") / extension_id + if entry.is_absolute() or ".." in entry.parts or not entry.is_relative_to(package): + raise ValueError("the installed agent entry point must stay inside its extension directory") + artifacts = source / package + if (source / "extensions").is_symlink() or artifacts.is_symlink(): + raise ValueError("the installed agent directory must not be a symbolic link") + if not (source / entry).is_file(): + raise ValueError("the installed agent entry point is missing") + for path in artifacts.rglob("*"): + if path.is_symlink() or not (path.is_file() or path.is_dir()): + raise ValueError("the installed agent artifacts must be regular files or directories") + shutil.copytree(artifacts, target / package) + with (target / "config.json").open("w") as stream: + json.dump({"extension": {"installed": {extension_id: registration}}}, stream, indent=2) + stream.write("\n") PY cwd: "~/working" - name: "copy the real config and extensions and remove all alpha opt-ins" + name: "copy only the installed agent registration and package without auth or alpha settings" post: - run: 'rm -rf -- "$HOME/working/azd-agents-t0-config-list-alpha-{instance}"' @@ -60,7 +76,7 @@ goals: - "Confirm the command exits with code 0 without a picker, login prompt, or project mutation." - "Confirm the output lists agents.managed_harness, agents.voice, and agents.long_running as blocks with Name, Description, Status: Off, and an AZD_FOUNDRY_EXT_ENABLE_AGENTS_* Environment variable line, in the layout of 'azd config list-alpha'." - "Run 'AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE=true azd ai agent config list-alpha'. Confirm only agents.voice shows Status: On." - - "Run 'azd ai agent config list-alpha --output json'. Confirm the output is a JSON object with a features array whose items have name, description, status, and envVar fields." + - "Run 'azd ai agent config list-alpha --output json'. Confirm the output is a JSON object with a features array whose items have name, description, status, and envVar fields, and every status is Off." - "Run 'azd ai agent config list-alpha --output raw'. Confirm it exits non-zero and names the supported formats." - "Run 'azd ai agent config --help' and 'azd ai agent config list-alpha --help'. Confirm both exit with code 0." - "Run 'azd ai agent feature list'. Confirm it does not list features." From c90134715fe2a92ac60bc9ed8bb7df7414573967 Mon Sep 17 00:00:00 2001 From: huimiu Date: Thu, 22 Oct 2026 14:34:10 +0800 Subject: [PATCH 10/10] test: shorten the offline listing scenario comment --- .../tier0/0.32-config-list-alpha.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml index 5a0eb4998eb..b16fca1b7f2 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml @@ -1,4 +1,4 @@ -# Tier 0: alpha feature listing with isolated extension data, no auth or Azure cost. +# Tier 0: isolated alpha feature listing, no auth or Azure cost. # cspell:ignore shutil rglob copytree name: "config-list-alpha" command: 'AZD_CONFIG_DIR="$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" azd ai agent config list-alpha'