diff --git a/cli/azd/extensions/azure.ai.agents/README.md b/cli/azd/extensions/azure.ai.agents/README.md index 52e47a76443..d0f42a7c6d3 100644 --- a/cli/azd/extensions/azure.ai.agents/README.md +++ b/cli/azd/extensions/azure.ai.agents/README.md @@ -32,6 +32,23 @@ Each preview needs its own `alpha.agents.` key or its `AZD_FOUNDRY_EXT_ENABLE_*` variable. Unrelated keys such as `agents.hosted` do not turn them on. Ordinary hosted and plain prompt agents need no opt-in. +### Check feature status + +`azd ai agent config list-alpha` lists each feature with its name, description, +status in the selected environment, and environment variable, in the layout of +`azd config list-alpha`. + +- `-e/--environment` selects another environment. An explicit empty environment is rejected. +- `--output json` sets the output format for scripts. + +```bash +azd ai agent config list-alpha +azd ai agent config list-alpha -e dev --output json +``` + +When a status cannot be read, that entry shows `Unknown` with the error. The +command still lists every feature and then exits with an error. + While a feature is off, its flags, commands, and help sections are hidden from help. This covers `invoke --long-running`, `invoke --no-wait`, the `invocations` group, `init --voice`, and the `prompt-voice` choice of diff --git a/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md b/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md index bd96d4ea15f..80c6da4f953 100644 --- a/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md +++ b/cli/azd/extensions/azure.ai.agents/docs/specs/preview-feature-flags.md @@ -17,6 +17,7 @@ Managed harness means `kind: prompt` with harness type `microsoft_copilot_previe azure.ai.agents/ ├── internal/ │ ├── cmd/ +│ │ ├── config.go # config list-alpha command │ │ ├── root.go # One pre-run check and the help hook │ │ └── preview_features.go # Annotations, enforcement, help hiding │ ├── project/ @@ -101,7 +102,7 @@ The `init` pre-run also binds `WithInitScope` before enforcement or discovery. I Enforcement has two layers. Each layer has one check. -**Command layer.** Commands and flags carry annotations that name a registered feature. The root `PersistentPreRunE` runs after the SDK pre-run, so `-e` and `AZD_ENVIRONMENT` are already applied. It calls `enforcePreviewFeatures` for the executing command with its args. Whenever the gate would reject, it first runs the command's local argument checks from `validateBeforePreview` so local conflicts are reported before `feature_not_enabled` or an empty environment error. These checks use only flags and args, with no source reads or state changes. The existing debug logging and telemetry steps keep their order. The gate makes no host reads for a command without a preview annotation or a set preview flag. A bound environment error only fails commands that perform a preview check. +**Command layer.** Commands and flags carry annotations that name a registered feature. The root `PersistentPreRunE` runs after the SDK pre-run, so `-e` and `AZD_ENVIRONMENT` are already applied. It calls `enforcePreviewFeatures` for the executing command with its args. Whenever the gate would reject, it first runs the command's local argument checks from `validateBeforePreview` so local conflicts are reported before `feature_not_enabled` or an empty environment error. These checks use only flags and args, with no source reads or state changes. The existing debug logging and telemetry steps keep their order. The gate makes no host reads for a command without a preview annotation or a set preview flag. A bound environment error only fails commands that perform a preview check or carry `rejectEmptyEnvironmentAnnotation`. | Annotation | Placed on | Effect | | --- | --- | --- | @@ -151,6 +152,14 @@ azd env set AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE true The [README](../../README.md#preview-feature-opt-ins) lists the equivalent commands for all three capabilities. +## Alpha feature listing + +`azd ai agent config list-alpha` mirrors `azd config list-alpha`. It prints every feature from `All` as a block with `Name`, `Description`, `Status` (`On` or `Off`), and `Environment variable`. It calls `IsEnabled` once per feature with the environment selected by `-e`, so the status matches what enforcement sees. It prints no source for an opt-in because the API returns only a decision. + +An explicit empty environment is rejected before any source is read. The root pre-run rejects it before the project read for telemetry. + +The default text output prints `Status: Unknown` and an `Error` line for a feature whose status cannot be read. `--output json` prints `{"features":[...]}` with `name`, `description`, `status`, `envVar`, and an `error` field when the read failed. Both formats list every feature before the command exits with an error that counts the failed reads. The command is read only and needs no authentication. + ## Documented enablement methods The documented ways to opt into a feature are listed below. These opt-ins are additive: any enabling value is sufficient. @@ -209,6 +218,6 @@ Disabled capabilities return the structured `feature_not_enabled` compatibility Tests cover source combinations, all supported boolean spellings and non-enabling values, nested per-feature alpha config, ignored `alpha.all` and unrelated keys, exact environment keys, explicit environment forwarding, optional scopes and sentinel compatibility, genuine failures despite positive sources, malformed responses, context forwarding, and snapshot isolation without input mutation. Registry tests cover declaration rules, the stable copy returned by `All`, and `Lookup`. -Command regressions run through the root command so the central pre-run applies. They cover which parsed flags and values require which feature, inherited command annotations, service gate placement, independent gates, obsolete hosted opt-in isolation, selected-environment forwarding for enforcement and help, help hiding per feature with a process fallback, hidden commands that stay reachable, staged manifests, local invocation, deployed-name fallback, unchanged inputs on rejection, ordinary flows, and retained read access. Help snapshots cover the full surface with every feature on and the default surface with every feature off. A walk over every command with every feature off rejects help text that names a disabled preview. +Command regressions run through the root command so the central pre-run applies. They cover which parsed flags and values require which feature, inherited command annotations, service gate placement, independent gates, obsolete hosted opt-in isolation, selected-environment forwarding for enforcement and help, help hiding per feature with a process fallback, hidden commands that stay reachable, staged manifests, local invocation, deployed-name fallback, unchanged inputs on rejection, ordinary flows, and retained read access. Help snapshots cover the full surface with every feature on and the default surface with every feature off. A walk over every command with every feature off rejects help text that names a disabled preview. Alpha listing tests cover text and JSON output, per-feature read errors, the selected environment, and the output flag allow list. -Run offline tests, build, vet, and scoped lint against the pinned SDK. Tier 0 scenarios cover the hidden and enabled help states. +Run offline tests, build, vet, and scoped lint against the pinned SDK. Tier 0 scenarios cover the hidden and enabled help states and `config list-alpha`. diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go new file mode 100644 index 00000000000..cf17dda3c7f --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config.go @@ -0,0 +1,186 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +package cmd + +import ( + "context" + "errors" + "fmt" + "io" + "strings" + + "azureaiagent/internal/featureflags" + + "github.com/azure/azure-dev/cli/azd/pkg/azdext" + "github.com/spf13/cobra" +) + +// Status values match `azd config list-alpha`. +const ( + alphaStatusOn = "On" + alphaStatusOff = "Off" + alphaStatusUnknown = "Unknown" +) + +// newConfigCommand returns the `azd ai agent config` parent command. +func newConfigCommand(extCtx *azdext.ExtensionContext) *cobra.Command { + extCtx = ensureExtensionContext(extCtx) + cmd := &cobra.Command{ + Use: "config", + Short: "Inspect agent extension configuration.", + Long: `Inspect agent extension configuration. + +Use 'azd config' to change azd user configuration.`, + Example: ` # Display the agent features in alpha stage + azd ai agent config list-alpha`, + } + cmd.AddCommand(newConfigListAlphaCommand(extCtx)) + return cmd +} + +// AlphaFeatureItem is the JSON contract for one agent alpha feature. +// Fields added later are additive. Consumers must ignore unknown fields. +type AlphaFeatureItem struct { + // Name is the feature ID used under alpha in user config. + Name string `json:"name"` + // Description is a short summary of the feature. + Description string `json:"description,omitempty"` + // Status is On or Off for the selected environment, or Unknown when + // Error is set. + Status string `json:"status"` + // EnvVar is the environment variable that turns the feature on. + EnvVar string `json:"envVar"` + // Error is set when the status could not be read. + Error string `json:"error,omitempty"` +} + +type alphaFeatureListResponse struct { + Features []AlphaFeatureItem `json:"features"` +} + +type alphaFeatureReader func(context.Context, featureflags.Feature) (bool, error) + +func newConfigListAlphaCommand(extCtx *azdext.ExtensionContext) *cobra.Command { + cmd := &cobra.Command{ + Use: "list-alpha", + Short: "Display the list of agent features in alpha stage.", + Long: `Display the list of agent features in alpha stage with the status of each +in the selected environment and the environment variable that turns it on. + +Use -e/--environment to read another azd environment. When a status cannot be +read, the entry reports the error and the command exits with an error after +listing every feature.`, + Example: ` # Display the agent features in alpha stage + azd ai agent config list-alpha + + # Display the features for another environment as JSON + azd ai agent config list-alpha -e dev --output json + + # Turn on a specific alpha feature + azd config set alpha. on`, + Annotations: map[string]string{rejectEmptyEnvironmentAnnotation: "true"}, + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, args []string) error { + ctx := cmd.Context() + client, err := azdext.NewAzdClient() + if err != nil { + err = fmt.Errorf("connecting to azd to read alpha features: %w", err) + } else { + defer client.Close() + } + environmentName := featureflags.Environment(ctx) + read := func(ctx context.Context, feature featureflags.Feature) (bool, error) { + if err != nil { + return false, err + } + return featureflags.IsEnabled(ctx, client, feature, environmentName) + } + return runConfigListAlpha(ctx, cmd.OutOrStdout(), extCtx.OutputFormat, read) + }, + } + azdext.RegisterFlagOptions(cmd, azdext.FlagOptions{ + Name: "output", + AllowedValues: []string{"json", "text"}, + Default: "text", + }) + return cmd +} + +// runConfigListAlpha reads every registered feature, writes the list, and +// then reports whether any status could not be read. +func runConfigListAlpha(ctx context.Context, out io.Writer, format string, read alphaFeatureReader) error { + features := featureflags.All() + items := make([]AlphaFeatureItem, 0, len(features)) + var failures []error + for _, feature := range features { + item := AlphaFeatureItem{ + Name: feature.ID, + Description: feature.Description, + Status: alphaStatusOff, + EnvVar: feature.EnvVar, + } + enabled, err := read(ctx, feature) + switch { + case err != nil: + item.Status = alphaStatusUnknown + item.Error = err.Error() + if errors.Is(err, context.Canceled) { + failures = append([]error{err}, failures...) + } else { + failures = append(failures, err) + } + case enabled: + item.Status = alphaStatusOn + } + items = append(items, item) + } + + var err error + if format == "json" { + err = writeJSON(out, alphaFeatureListResponse{Features: items}) + } else { + err = writeAlphaFeatures(out, items) + } + if err != nil { + return err + } + if len(failures) > 0 { + return &alphaReadError{total: len(items), causes: failures} + } + return nil +} + +// alphaReadError reports how many statuses could not be read. The listing +// already shows each message, so Error stays short while Unwrap keeps the +// read errors for classification. Cancellation is listed first. +type alphaReadError struct { + total int + causes []error +} + +func (e *alphaReadError) Error() string { + return fmt.Sprintf("could not read the status of %d of %d alpha features", len(e.causes), e.total) +} + +func (e *alphaReadError) Unwrap() []error { return e.causes } + +// writeAlphaFeatures writes one block per feature in the layout of +// `azd config list-alpha`, plus the environment variable. +func writeAlphaFeatures(out io.Writer, items []AlphaFeatureItem) error { + blocks := make([]string, 0, len(items)) + for _, item := range items { + lines := []string{ + "Name: " + item.Name, + "Description: " + item.Description, + "Status: " + item.Status, + "Environment variable: " + item.EnvVar, + } + if item.Error != "" { + lines = append(lines, "Error: "+item.Error) + } + blocks = append(blocks, strings.Join(lines, "\n")) + } + _, err := fmt.Fprintln(out, strings.Join(blocks, "\n\n")) + return err +} diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go new file mode 100644 index 00000000000..281080c3752 --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/config_test.go @@ -0,0 +1,320 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "io" + "strings" + "testing" + + "azureaiagent/internal/exterrors" + "azureaiagent/internal/featureflags" + + "github.com/azure/azure-dev/cli/azd/pkg/azdext" + "github.com/stretchr/testify/require" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" +) + +func TestConfigListAlphaText(t *testing.T) { + var out bytes.Buffer + err := runConfigListAlpha(t.Context(), &out, "text", + func(_ context.Context, feature featureflags.Feature) (bool, error) { + return feature == featureflags.Voice, nil + }) + require.NoError(t, err) + require.Equal(t, strings.Join([]string{ + "Name: agents.managed_harness", + "Description: Prompt agents that run on a Copilot harness", + "Status: Off", + "Environment variable: AZD_FOUNDRY_EXT_ENABLE_AGENTS_MANAGED_HARNESS", + "", + "Name: agents.voice", + "Description: Voice, prompt-voice, and hosted voice agents", + "Status: On", + "Environment variable: AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE", + "", + "Name: agents.long_running", + "Description: Long-running Responses invocations", + "Status: Off", + "Environment variable: AZD_FOUNDRY_EXT_ENABLE_AGENTS_LONG_RUNNING", + "", + }, "\n"), out.String()) +} + +func TestConfigListAlphaJSON(t *testing.T) { + var out bytes.Buffer + err := runConfigListAlpha(t.Context(), &out, "json", + func(_ context.Context, feature featureflags.Feature) (bool, error) { + return feature == featureflags.LongRunning, nil + }) + require.NoError(t, err) + var response alphaFeatureListResponse + require.NoError(t, json.Unmarshal(out.Bytes(), &response)) + require.Len(t, response.Features, len(featureflags.All())) + for i, feature := range featureflags.All() { + item := response.Features[i] + require.Equal(t, feature.ID, item.Name) + require.Equal(t, feature.EnvVar, item.EnvVar) + require.Equal(t, feature.Description, item.Description) + want := "Off" + if feature == featureflags.LongRunning { + want = "On" + } + require.Equal(t, want, item.Status) + require.Empty(t, item.Error) + } + require.NotContains(t, out.String(), "error") +} + +func TestConfigListAlphaReportsEachReadError(t *testing.T) { + for _, format := range []string{"text", "json"} { + t.Run(format, func(t *testing.T) { + var out bytes.Buffer + err := runConfigListAlpha(t.Context(), &out, format, + func(_ context.Context, feature featureflags.Feature) (bool, error) { + if feature == featureflags.Voice { + return false, errors.New("cannot read selected environment") + } + return true, nil + }) + require.ErrorContains(t, err, "1 of 3") + text := out.String() + for _, feature := range featureflags.All() { + require.Contains(t, text, feature.ID, "every feature is listed despite the error") + } + require.Contains(t, text, "cannot read selected environment") + if format == "json" { + var response alphaFeatureListResponse + require.NoError(t, json.Unmarshal(out.Bytes(), &response)) + for _, item := range response.Features { + if item.Name == featureflags.Voice.ID { + require.Equal(t, "Unknown", item.Status) + require.Equal(t, "cannot read selected environment", item.Error) + } else { + require.Equal(t, "On", item.Status) + require.Empty(t, item.Error) + } + } + } + if format == "text" { + require.Contains(t, text, "Name: agents.voice\n"+ + "Description: Voice, prompt-voice, and hosted voice agents\n"+ + "Status: Unknown\n"+ + "Environment variable: AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE\n"+ + "Error: cannot read selected environment") + require.Contains(t, text, "Name: agents.long_running\n"+ + "Description: Long-running Responses invocations\nStatus: On") + } + }) + } +} + +func TestConfigListAlphaKeepsReadErrorChain(t *testing.T) { + notFound := status.Error(codes.NotFound, "environment not found") + err := runConfigListAlpha(t.Context(), io.Discard, "text", + func(_ context.Context, feature featureflags.Feature) (bool, error) { + switch feature { + case featureflags.Voice: + return false, notFound + case featureflags.LongRunning: + return false, context.Canceled + } + return false, nil + }) + require.ErrorContains(t, err, "2 of 3") + require.ErrorIs(t, err, context.Canceled) + require.ErrorIs(t, err, notFound) + require.Equal(t, "could not read the status of 2 of 3 alpha features", err.Error()) + readErr, ok := errors.AsType[*alphaReadError](err) + require.True(t, ok) + require.ErrorIs(t, readErr.Unwrap()[0], context.Canceled, "cancellation takes precedence") +} + +func TestConfigListAlphaCommandReadsSelectedEnvironment(t *testing.T) { + clearPreviewProcess(t) + t.Chdir(t.TempDir()) + _, env := startPreviewCommandServer(t, map[string]any{ + "all": "on", + "agents": map[string]any{"voice": "on"}, + }, &azdext.ProjectConfig{Path: t.TempDir()}, nil) + env.mu.Lock() + env.values = []*azdext.KeyValue{{Key: featureflags.LongRunning.EnvVar, Value: "true"}} + env.mu.Unlock() + + text, err := executeRoot(t, "config", "list-alpha", "-e", "selected-env", "--output", "json") + require.NoError(t, err) + var response alphaFeatureListResponse + require.NoError(t, json.Unmarshal([]byte(text), &response)) + states := map[string]string{} + for _, item := range response.Features { + states[item.Name] = item.Status + } + require.Equal(t, map[string]string{ + "agents.managed_harness": "Off", "agents.voice": "On", "agents.long_running": "On", + }, states) + env.mu.Lock() + require.Equal(t, "selected-env", env.selected) + env.mu.Unlock() + + t.Setenv(featureflags.ManagedHarness.EnvVar, "true") + text, err = executeRoot(t, "config", "list-alpha", "-e", "selected-env") + require.NoError(t, err) + require.Contains(t, text, "Name: agents.managed_harness\n"+ + "Description: Prompt agents that run on a Copilot harness\nStatus: On") + require.NotContains(t, strings.ToLower(text), "azure.yaml") + require.NotContains(t, strings.ToLower(text), "source") +} + +func TestConfigListAlphaCommandUsesProcessValue(t *testing.T) { + clearPreviewProcess(t) + t.Setenv(featureflags.Voice.EnvVar, "1") + startPreviewCommandServer(t, nil, nil, nil) + text, err := executeRoot(t, "config", "list-alpha") + require.NoError(t, err) + require.Contains(t, text, "Name: agents.voice\n"+ + "Description: Voice, prompt-voice, and hosted voice agents\nStatus: On") + require.Contains(t, text, "Name: agents.long_running\n"+ + "Description: Long-running Responses invocations\nStatus: Off") +} + +func TestConfigListAlphaCommandReportsConstructionError(t *testing.T) { + for _, format := range []string{"text", "json"} { + t.Run(format, func(t *testing.T) { + clearPreviewProcess(t) + t.Setenv("AZD_SERVER", "dns:///%zz") + client, cause := azdext.NewAzdClient() + if client != nil { + t.Cleanup(client.Close) + } + require.Error(t, cause, "the real constructor must reject the target") + require.Nil(t, client) + + text, err := executeRoot(t, "config", "list-alpha", "--output", format) + require.Error(t, err) + readErr, ok := errors.AsType[*alphaReadError](err) + require.True(t, ok, "%v", err) + features := featureflags.All() + require.Len(t, readErr.Unwrap(), len(features)) + wrapped := readErr.Unwrap()[0] + original := errors.Unwrap(wrapped) + require.Error(t, original) + require.EqualError(t, original, cause.Error()) + require.EqualError(t, wrapped, "connecting to azd to read alpha features: "+cause.Error()) + require.ErrorIs(t, err, original) + for _, failure := range readErr.Unwrap() { + require.ErrorIs(t, failure, original) + } + if format == "json" { + var response alphaFeatureListResponse + require.NoError(t, json.Unmarshal([]byte(text), &response)) + require.Len(t, response.Features, len(features)) + for i, feature := range features { + item := response.Features[i] + require.Equal(t, feature.ID, item.Name) + require.Equal(t, feature.Description, item.Description) + require.Equal(t, feature.EnvVar, item.EnvVar) + require.Equal(t, alphaStatusUnknown, item.Status) + require.Equal(t, wrapped.Error(), item.Error) + } + } else { + require.Equal(t, len(features), strings.Count(text, "Status: Unknown\n")) + require.Equal(t, len(features), strings.Count(text, "Error: "+wrapped.Error())) + for _, feature := range features { + require.Contains(t, text, "Name: "+feature.ID+"\n") + require.Contains(t, text, "Environment variable: "+feature.EnvVar+"\n") + } + } + }) + } +} + +type alphaFailingWriter struct { + err error +} + +func (w alphaFailingWriter) Write([]byte) (int, error) { + return 0, w.err +} + +func TestConfigListAlphaCommandConstructionErrorPreservesWriteError(t *testing.T) { + for _, format := range []string{"text", "json"} { + t.Run(format, func(t *testing.T) { + clearPreviewProcess(t) + t.Setenv("AZD_SERVER", "dns:///%zz") + writeErr := errors.New("cannot write alpha features") + root := NewRootCommand() + root.SetOut(alphaFailingWriter{err: writeErr}) + root.SetErr(io.Discard) + root.SetArgs([]string{"config", "list-alpha", "--output", format}) + require.ErrorIs(t, root.Execute(), writeErr) + }) + } +} + +func TestConfigListAlphaCommandValidatesBeforeConstruction(t *testing.T) { + for _, args := range [][]string{ + {"--output", "raw"}, + {"--environment", ""}, + {"--environment", " \t "}, + } { + t.Run(strings.Join(args, "/"), func(t *testing.T) { + clearPreviewProcess(t) + t.Setenv("AZD_SERVER", "dns:///%zz") + text, err := executeRoot(t, append([]string{"config", "list-alpha"}, args...)...) + require.Error(t, err) + require.NotContains(t, err.Error(), "connecting to azd") + require.NotContains(t, err.Error(), "could not read the status") + require.Empty(t, text) + }) + } +} + +func TestConfigListAlphaRejectsUnsupportedOutput(t *testing.T) { + clearPreviewProcess(t) + _, err := executeRoot(t, "config", "list-alpha", "--output", "raw") + require.ErrorContains(t, err, "raw") +} + +func TestConfigListAlphaRejectsEmptyEnvironment(t *testing.T) { + for _, value := range []string{"", " \t "} { + for _, flag := range []string{"--environment", "-e"} { + t.Run(flag+"/"+value, func(t *testing.T) { + clearPreviewProcess(t) + t.Setenv("AZD_ENVIRONMENT", "default-env") + _, env := startPreviewCommandServer(t, nil, nil, nil) + + text, err := executeRoot(t, "config", "list-alpha", flag, value) + localErr, ok := errors.AsType[*azdext.LocalError](err) + require.True(t, ok, "%v", err) + require.Equal(t, exterrors.CodeInvalidParameter, localErr.Code) + require.Equal(t, azdext.LocalErrorCategoryValidation, localErr.Category) + require.Equal(t, "the --environment flag was set to an empty value", localErr.Message) + require.Empty(t, text) + require.Zero(t, env.reads.Load(), "no project read or telemetry report should reach the host") + }) + } + } +} + +func TestConfigListAlphaHelpAcceptsEmptyEnvironment(t *testing.T) { + clearPreviewProcess(t) + _, env := startPreviewCommandServer(t, nil, nil, nil) + + text, err := executeRoot(t, "config", "list-alpha", "-e", "", "--help") + require.NoError(t, err) + require.Contains(t, text, "Display the list of agent features in alpha stage") + require.Zero(t, env.reads.Load()) +} + +func TestFeatureCommandRemoved(t *testing.T) { + clearPreviewProcess(t) + for _, child := range NewRootCommand().Commands() { + require.NotEqual(t, "feature", child.Name()) + } +} diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/json_output.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/json_output.go index 311801e48be..5dbb54cd36b 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/json_output.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/json_output.go @@ -6,13 +6,20 @@ package cmd import ( "encoding/json" "fmt" + "io" + "os" ) func emitJSON(value any) error { + return writeJSON(os.Stdout, value) +} + +// writeJSON writes value as indented JSON followed by a newline. +func writeJSON(w io.Writer, value any) error { data, err := json.MarshalIndent(value, "", " ") if err != nil { return fmt.Errorf("failed to marshal JSON output: %w", err) } - fmt.Println(string(data)) - return nil + _, err = fmt.Fprintln(w, string(data)) + return err } diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go index 61506da4962..3b92e447813 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go @@ -14,6 +14,8 @@ import ( "github.com/spf13/cobra" ) +const rejectEmptyEnvironmentAnnotation = "azure.ai.agents/reject-empty-environment" + func NewRootCommand() *cobra.Command { rootCmd, extCtx := azdext.NewExtensionRootCommand(azdext.ExtensionCommandOptions{ Name: "agent", @@ -52,6 +54,11 @@ func NewRootCommand() *cobra.Command { if err := enforcePreviewFeatures(cmd, args, extCtx.Environment); err != nil { return err } + if cmd.Annotations[rejectEmptyEnvironmentAnnotation] == "true" { + if err := featureflags.EnvironmentError(cmd.Context()); err != nil { + return err + } + } operation := telemetryOperation(cmd.CommandPath()) switch operation { case "deploy", "init", "listen", "metadata", "version": @@ -111,6 +118,7 @@ func NewRootCommand() *cobra.Command { rootCmd.AddCommand(newStateStoresCommand(extCtx)) rootCmd.AddCommand(newInvocationsCommand(extCtx)) rootCmd.AddCommand(newSampleCommand(extCtx)) + rootCmd.AddCommand(newConfigCommand(extCtx)) rootCmd.AddCommand(newDoctorCommand()) rootCmd.AddCommand(newCodeCommand(extCtx)) diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go b/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go index c025bc294f2..d6c28318915 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/root_test.go @@ -9,9 +9,28 @@ import ( "strings" "testing" + "github.com/spf13/cobra" "github.com/stretchr/testify/require" ) +func TestRootCommandInvokeAcceptsEmptyEnvironment(t *testing.T) { + clearPreviewProcess(t) + _, env := startPreviewCommandServer(t, nil, nil, nil) + root := NewRootCommand() + invoke, _, err := root.Find([]string{"invoke"}) + require.NoError(t, err) + ran := false + invoke.RunE = func(*cobra.Command, []string) error { + ran = true + return nil + } + root.SetArgs([]string{"invoke", "-e", "", "hello"}) + + require.NoError(t, root.ExecuteContext(t.Context())) + require.True(t, ran) + require.Positive(t, env.reads.Load(), "project telemetry should still read the project") +} + func TestRootCommand_PublicPreviewCommandsVisible(t *testing.T) { cmd := NewRootCommand() @@ -25,6 +44,7 @@ func TestRootCommand_PublicPreviewCommandsVisible(t *testing.T) { for _, name := range []string{ "add", "code", + "config", "connection", "delete", "doctor", diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config.snap b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config.snap new file mode 100644 index 00000000000..9e015209256 --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config.snap @@ -0,0 +1,23 @@ +Inspect agent extension configuration. + +Use 'azd config' to change azd user configuration. + +Usage + azd ai agent config [command] + +Available Commands + list-alpha : Display the list of agent features in alpha stage. + +Global Flags + -C, --cwd string : Sets the current working directory + --debug : Enables debug and diagnostics logging + -e, --environment string : The name of the environment to use + -h, --help : help for config + --no-prompt : Runs without prompts. Uses existing values; fails if any required value or decision cannot be resolved automatically. + -o, --output string : The output format (default "default") + +Examples + Display the agent features in alpha stage + azd ai agent config list-alpha + +Use azd ai agent config [command] --help to view examples and more information about a specific command. diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config_list-alpha.snap b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config_list-alpha.snap new file mode 100644 index 00000000000..d2c8ee50e39 --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-config_list-alpha.snap @@ -0,0 +1,27 @@ +Display the list of agent features in alpha stage with the status of each +in the selected environment and the environment variable that turns it on. + +Use -e/--environment to read another azd environment. When a status cannot be +read, the entry reports the error and the command exits with an error after +listing every feature. + +Usage + azd ai agent config list-alpha [flags] + +Global Flags + -C, --cwd string : Sets the current working directory + --debug : Enables debug and diagnostics logging + -e, --environment string : The name of the environment to use + -h, --help : help for list-alpha + --no-prompt : Runs without prompts. Uses existing values; fails if any required value or decision cannot be resolved automatically. + -o, --output string : The output format (supported: json, text) (default "text") + +Examples + Display the agent features in alpha stage + azd ai agent config list-alpha + + Display the features for another environment as JSON + azd ai agent config list-alpha -e dev --output json + + Turn on a specific alpha feature + azd config set alpha. on diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap index c2ae7c29ca6..68f8b4c594a 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap @@ -17,6 +17,7 @@ Usage Available Commands add : Add a file-backed Agent service to the current azd project. code : Manage agent source code. (Preview) + config : Inspect agent extension configuration. connection : Manage connection service dependencies for an agent. delete : Delete an agent. doctor : Diagnose problems with an azd ai agent project. diff --git a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelpPreviewsOff-root.snap b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelpPreviewsOff-root.snap index b7675d946d6..1c400fa52e5 100644 --- a/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelpPreviewsOff-root.snap +++ b/cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelpPreviewsOff-root.snap @@ -17,6 +17,7 @@ Usage Available Commands add : Add a file-backed Agent service to the current azd project. code : Manage agent source code. (Preview) + config : Inspect agent extension configuration. connection : Manage connection service dependencies for an agent. delete : Delete an agent. doctor : Diagnose problems with an azd ai agent project. diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md index 44eab4c48bb..7d05a21dd02 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/README.md @@ -345,6 +345,7 @@ in any order, any time. | `tier0/0.28-agent-add-multiple-projects.yaml` | `agent add` rejects multiple Project services before mutation | | `tier0/0.29-agent-add-path-safety.yaml` | `agent add` rejects definition and source paths outside the project | | `tier0/0.30-agent-add-wrong-host-collision.yaml` | `agent add` rejects a target service key owned by a non-Agent host without mutating `azure.yaml` | +| `tier0/0.32-config-list-alpha.yaml` | `config list-alpha` text and JSON output with isolated agent registration/artifacts and no inherited alpha opt-ins | The invocation lifecycle scenarios above are offline help/validation checks, not live execution tests. They do not require a deployed long-running agent or add Tier 2 provisioning dependencies. Actual HTTP @@ -519,7 +520,7 @@ grouping — colons are treated as ordinary characters by the filter): | Namespace | Values | Meaning | |---|---|---| | `tier:N` | `tier:0`, `tier:1`, `tier:1b`, `tier:2` | The tier the scenario belongs to (same axis as the directory's four sections above). Use this to express cost / auth profile in one tag. | -| `cmd:*` | `cmd:init`, `cmd:show`, `cmd:invoke`, `cmd:invocations`, `cmd:sessions`, `cmd:files`, `cmd:state-stores`, `cmd:monitor`, `cmd:endpoint`, `cmd:run`, `cmd:doctor`, `cmd:eval`, `cmd:optimize`, `cmd:sample`, `cmd:down`, `cmd:provision`, `cmd:deploy`, `cmd:version`, `cmd:help`, `cmd:code`, `cmd:delete`, `cmd:toolbox`, `cmd:connection`, `cmd:add` | The top-level `azd ai agent` (or `azd`) command(s) the scenario exercises. Multi-command scenarios (e.g. `2.12-run-local-and-invoke-local` runs both `run` and `invoke --local`; `2.00-setup` runs `init` + `provision` + `deploy`) carry multiple `cmd:*` tags. `cmd:add` tags the Agent add command; `cmd:toolbox` and `cmd:connection` cover Agent dependency composition, not the sibling extensions' resource lifecycle commands. | +| `cmd:*` | `cmd:init`, `cmd:show`, `cmd:invoke`, `cmd:invocations`, `cmd:sessions`, `cmd:files`, `cmd:state-stores`, `cmd:monitor`, `cmd:endpoint`, `cmd:run`, `cmd:doctor`, `cmd:eval`, `cmd:optimize`, `cmd:sample`, `cmd:down`, `cmd:provision`, `cmd:deploy`, `cmd:version`, `cmd:help`, `cmd:code`, `cmd:delete`, `cmd:toolbox`, `cmd:connection`, `cmd:add`, `cmd:config` | The top-level `azd ai agent` (or `azd`) command(s) the scenario exercises. Multi-command scenarios (e.g. `2.12-run-local-and-invoke-local` runs both `run` and `invoke --local`; `2.00-setup` runs `init` + `provision` + `deploy`) carry multiple `cmd:*` tags. `cmd:add` tags the Agent add command; `cmd:toolbox` and `cmd:connection` cover Agent dependency composition, not the sibling extensions' resource lifecycle commands. | | traits | `parallel-safe`, `serial-only`, `negative-path`, `picker`, `verify-deploy`, `manual-seed` | `parallel-safe` ↔ `serial-only` are mutually exclusive: all Tier 0 / Tier 1 / Tier 1b scenarios are `parallel-safe`, all Tier 2 are `serial-only`. `negative-path` flags arg-/CLI-validation scenarios that assert errors or non-zero exit codes rather than happy-path success. `picker` flags scenarios whose primary purpose is exercising interactive picker UX. `verify-deploy` flags Tier 1b scenarios that verify a Tier 1 scaffold deploys. `manual-seed` flags an opt-in scenario that needs an externally prepared fixture before it runs; filtering by this tag does not create the fixture. | The tool returns matching `manual-seed` rows, but runner plans exclude them @@ -690,6 +691,10 @@ How they're used here: (`1.04-init-from-code`, `1.06-init-deploy-mode-code`) also copy a committed Python fixture into the dir so the source exists before the wizard's "Use the code in the current directory" flow inspects it (see [Fixtures](#fixtures)). +- `0.32` creates a minimal config containing only the installed `azure.ai.agents` registration + and copies that extension's package, including supporting files. It does not copy auth state, + other extensions, alpha settings, or unrelated config. Missing registrations/artifacts and + symbolic links fail setup explicitly. A `post` hook removes the dedicated working directory. - **`pre` gh-auth guard** — the remote azure.yaml project scenarios (`1.03-init-from-azure-yaml-url`, `1.05-init-flag-agent-name`) run `gh auth status` and fail fast if GitHub CLI isn't authenticated, because downloading the project file and its sibling @@ -800,7 +805,8 @@ be run back to back in any order within a tier: - Tier 0/1 stateful scenarios **pre-wipe** their own `cwd`. Cleanup is pre-wipe **only** (no `post` delete), so the generated scaffold stays on disk for - inspection after a run while the next run still starts clean. + inspection after a run while the next run still starts clean. Scenario `0.32` also removes + its config copy and working directory with a `post` hook. - Tier 1b scenarios use the prerequisite worker's verified `scaffold_dir` and preserve that Tier 1 scaffold, but use an always-run `post` hook to down their isolated Azure resources. Cleanup failure is a separate scenario failure and leaves the local environment available diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml index d2f9e26b8bf..4336adf9415 100644 --- a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml @@ -12,7 +12,7 @@ env: goals: - "Wait for the help output to render (it includes an ASCII-art banner and a 'Usage' section)." - - "Confirm the 'Available Commands' list includes: code, delete, doctor, endpoint, eval, files, init, invocations, invoke, monitor, optimize, pack, publish, run, sample, sessions, show, version." + - "Confirm the 'Available Commands' list includes: code, config, delete, doctor, endpoint, eval, files, init, invocations, invoke, monitor, optimize, pack, publish, run, sample, sessions, show, version." - "Confirm the global flags are listed: --cwd/-C, --debug, --environment/-e, --no-prompt, --output/-o." - "Confirm the Environments & Environment Variables section explains .azure//.env, azd env new/select/set/get-values, FOUNDRY_PROJECT_ENDPOINT, AGENT_ deployment values, and the service-level env mapping for runtime settings." - "Compare root, init, and files list help: headings should be consistently bold and underlined, command examples and flags highlighted, and usage paths start with azd ai agent. With NO_COLOR=1, the same content must remain readable without escape sequences." diff --git a/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml new file mode 100644 index 00000000000..b16fca1b7f2 --- /dev/null +++ b/cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.32-config-list-alpha.yaml @@ -0,0 +1,88 @@ +# Tier 0: isolated alpha feature listing, no auth or Azure cost. +# cspell:ignore shutil rglob copytree +name: "config-list-alpha" +command: 'AZD_CONFIG_DIR="$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" azd ai agent config list-alpha' +cwd: "~/working/azd-agents-t0-config-list-alpha-{instance}" +tags: ["tier:0", "cmd:config", "cmd:help", "parallel-safe"] + +env: + AZD_FOUNDRY_EXT_ENABLE_AGENTS_MANAGED_HARNESS: "" + AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE: "" + AZD_FOUNDRY_EXT_ENABLE_AGENTS_LONG_RUNNING: "" + AZURE_ENV_NAME: "" + AZURE_DEV_COLLECT_TELEMETRY: "no" + AZD_SKIP_UPDATE_CHECK: "true" + NO_COLOR: "1" + +pre: + - run: >- + rm -rf -- "$HOME/working/azd-agents-t0-config-list-alpha-{instance}" && + mkdir -p -- "$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" + cwd: "~/working" + name: "reset the dedicated offline directory" + - run: | + set -e + if [ -n "$AZD_CONFIG_DIR" ]; then + source_config="$AZD_CONFIG_DIR" + else + source_config="$HOME/.azd" + fi + python3 - "$source_config" "$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" <<'PY' + import json + import pathlib + import shutil + import sys + + source = pathlib.Path(sys.argv[1]) + target = pathlib.Path(sys.argv[2]) + with (source / "config.json").open() as stream: + data = json.load(stream) + extension_id = "azure.ai.agents" + installed = data.get("extension", {}).get("installed", {}) + if extension_id not in installed: + raise ValueError("install azure.ai.agents before running this scenario") + registration = installed[extension_id] + entry = pathlib.Path(registration["path"]) + package = pathlib.Path("extensions") / extension_id + if entry.is_absolute() or ".." in entry.parts or not entry.is_relative_to(package): + raise ValueError("the installed agent entry point must stay inside its extension directory") + artifacts = source / package + if (source / "extensions").is_symlink() or artifacts.is_symlink(): + raise ValueError("the installed agent directory must not be a symbolic link") + if not (source / entry).is_file(): + raise ValueError("the installed agent entry point is missing") + for path in artifacts.rglob("*"): + if path.is_symlink() or not (path.is_file() or path.is_dir()): + raise ValueError("the installed agent artifacts must be regular files or directories") + shutil.copytree(artifacts, target / package) + with (target / "config.json").open("w") as stream: + json.dump({"extension": {"installed": {extension_id: registration}}}, stream, indent=2) + stream.write("\n") + PY + cwd: "~/working" + name: "copy only the installed agent registration and package without auth or alpha settings" + +post: + - run: 'rm -rf -- "$HOME/working/azd-agents-t0-config-list-alpha-{instance}"' + cwd: "~/working" + name: "remove the dedicated config copy and working directory" + +goals: + - >- + Run every listed product command with the same + AZD_CONFIG_DIR="$HOME/working/azd-agents-t0-config-list-alpha-{instance}/azd-config" + prefix, including the voice process-variable check and help commands. + Keep this working directory free of any project. + - "Confirm the command exits with code 0 without a picker, login prompt, or project mutation." + - "Confirm the output lists agents.managed_harness, agents.voice, and agents.long_running as blocks with Name, Description, Status: Off, and an AZD_FOUNDRY_EXT_ENABLE_AGENTS_* Environment variable line, in the layout of 'azd config list-alpha'." + - "Run 'AZD_FOUNDRY_EXT_ENABLE_AGENTS_VOICE=true azd ai agent config list-alpha'. Confirm only agents.voice shows Status: On." + - "Run 'azd ai agent config list-alpha --output json'. Confirm the output is a JSON object with a features array whose items have name, description, status, and envVar fields, and every status is Off." + - "Run 'azd ai agent config list-alpha --output raw'. Confirm it exits non-zero and names the supported formats." + - "Run 'azd ai agent config --help' and 'azd ai agent config list-alpha --help'. Confirm both exit with code 0." + - "Run 'azd ai agent feature list'. Confirm it does not list features." + - "Confirm the output shows no field or text that names where a feature was turned on." + - >- + Confirm the real config directory is unchanged, using the caller's nonempty AZD_CONFIG_DIR + or otherwise $HOME/.azd. No alpha key may be added or removed in its config.json. + - "Take a screenshot of the text and JSON output." + - "Report a finding if a registered feature is missing, a status is wrong, or the command prompts or mutates a project."