bnxt_re/lib: Support direct verbs by sbasavapatna · Pull Request #1654 · linux-rdma/rdma-core

sbasavapatna · 2025-11-03T10:48:27Z

Hi,

This patchset supports Direct Verbs in the bnxt_re library.

This is required by vendor specific applications that need to manage
the HW resources directly and to implement the datapath in the
application.

To support this, the library and the driver are being enhanced to
provide Direct Verbs using which the application can allocate and
manage the HW resources (Queues, Doorbell etc) . The Direct Verbs
enable the application to implement the control path.

Patch#1 Update kernel headers
Patch#2 Add a helper for VA-backed buffer descriptors
Patch#3 Add ibv_buf APIs for user allocated memory
Patch#4 Export HSI file
Patch#5 Support DBR verbs
Patch#6 Update bnxt_re_cq to include struct verbs_cq
Patch#7 Support CQ verbs
Patch#8 Support QP verbs
Patch#9 Add man pages

Thanks,
-Harsha

jgunthorpe · 2025-12-17T20:01:19Z

Please take the debug_file stuff out of here, I'm not keen on every driver implementing their own stuff. We already have tracing frameworks supported, why not use one?

jgunthorpe · 2025-12-17T20:02:31Z

+			      struct bnxt_re_dv_db_region_attr *attr);
+int bnxt_re_dv_get_default_db_region(struct ibv_context *ibvctx,
+				     struct bnxt_re_dv_db_region_attr *out);
+void *bnxt_re_dv_umem_reg(struct ibv_context *ibvctx,


void *? That shouldn't be. Why do you have these functions when there is no kernel side?

sbasavapatna · 2025-12-24T04:22:02Z

On Thu, Dec 18, 2025 at 1:31 AM Jason Gunthorpe ***@***.***> wrote: *jgunthorpe* left a comment (linux-rdma/rdma-core#1654) <#1654 (comment)> Please take the debug_file stuff out of here, I'm not keen on every driver implementing their own stuff. We already have tracing frameworks supported, why not use one?

Deleted the debug log patch from the series and pushed the next version. Thanks, -Harsha

…

— Reply to this email directly, view it on GitHub <#1654 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTY2BJYE3NGLRRWOYN2OD34CGZCXAVCNFSM6AAAAACK7AEXQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMNRWHE2DSNBQHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

sbasavapatna · 2025-12-24T04:23:23Z

On Thu, Dec 18, 2025 at 1:32 AM Jason Gunthorpe ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In providers/bnxt_re/bnxt_re_dv.h <#1654 (comment)> : > +struct bnxt_re_dv_umem_reg_attr { + void *addr; + size_t size; + uint32_t access_flags; + uint64_t pgsz_bitmap; + uint64_t comp_mask; + int dmabuf_fd; +}; + +struct bnxt_re_dv_db_region_attr * +bnxt_re_dv_alloc_db_region(struct ibv_context *ctx); +int bnxt_re_dv_free_db_region(struct ibv_context *ctx, + struct bnxt_re_dv_db_region_attr *attr); +int bnxt_re_dv_get_default_db_region(struct ibv_context *ibvctx, + struct bnxt_re_dv_db_region_attr *out); +void *bnxt_re_dv_umem_reg(struct ibv_context *ibvctx, void *? That shouldn't be. Why do you have these functions when there is no kernel side?

It is a void *, since we want to pass an opaque handle to the application. This handle is passed back into the library during object creation. The application provides this handle along with an offset into this memory area to create the object (see CQ/QP creation DV APIs and commit messages). Thanks, -Harsha

…

— Reply to this email directly, view it on GitHub <#1654 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTY2BPWWJYFCN6B4PI3OWT4CGZGZAVCNFSM6AAAAACK7AEXQ2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTKOBZGQZTKMBUGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

jgunthorpe · 2026-01-07T19:58:32Z

So give it an opaque type, don't just randomly use void *s.

Again,why is this stuff here when there is no kernel side for umems?

sbasavapatna · 2026-01-08T07:05:36Z

On Thu, Jan 8, 2026 at 1:28 AM Jason Gunthorpe ***@***.***> wrote: *jgunthorpe* left a comment (linux-rdma/rdma-core#1654) <#1654 (comment)> So give it an opaque type, don't just randomly use void *s.

Are you suggesting to use a pointer to the forward declaration of that structure? Is that what you meant by "give it an opaque type"? Please clarify.

Again,why is this stuff here when there is no kernel side for umems?

The application registers the addr and the total size of the memory area with the library, initially using this API. Later during each object creation, the library uses this information to provide the actual va (base-addr + offset) for mapping/pinning, to the kernel driver. This has been captured in the kernel driver review thread: ***@***.***/

…

— Reply to this email directly, view it on GitHub <#1654 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTY2BIR6EFHFZXC2CENTI34FVQP5AVCNFSM6AAAAACK7AEXQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMRQGUZTOMBUGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

sbasavapatna · 2026-01-08T17:38:19Z

On Thu, Jan 8, 2026 at 1:28 AM Jason Gunthorpe @.> wrote: jgunthorpe left a comment (linux-rdma/rdma-core#1654) <#1654 (comment)> So give it an opaque type, don't just randomly use void s.
Are you suggesting to use a pointer to the forward declaration of that structure? Is that what you meant by "give it an opaque type"? Please clarify.
Again,why is this stuff here when there is no kernel side for umems?
The application registers the addr and the total size of the memory area with the library, initially using this API. Later during each object creation, the library uses this information to provide the actual va (base-addr + offset) for mapping/pinning, to the kernel driver. This has been captured in the kernel driver review thread: @./
…
— Reply to this email directly, view it on GitHub <#1654 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTY2BIR6EFHFZXC2CENTI34FVQP5AVCNFSM6AAAAACK7AEXQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMRQGUZTOMBUGI . You are receiving this because you authored the thread.Message ID: @.**>

Here's the pointer to the kernel review thread for reference:
https://lore.kernel.org/linux-rdma/CAHHeUGXGTfsK66DOGdE6Y5VYVaOR=1YA6b2inds6_EkPSwTBuA@mail.gmail.com/

jgunthorpe · 2026-01-09T18:13:40Z

Just have some 'struct bnxt_umem' as a forward declaration that never sees an actual definition. It is now an opaque type you can use instead of void.

You also need to write man pages for these functions, see providers/mlx5/man/mlx5dv_devx_umem_reg.3.md for instance.

sbasavapatna · 2026-01-13T17:29:01Z

On Fri, Jan 9, 2026 at 11:44 PM Jason Gunthorpe ***@***.***> wrote: *jgunthorpe* left a comment (linux-rdma/rdma-core#1654) <#1654 (comment)> Just have some 'struct bnxt_umem' as a forward declaration that never sees an actual definition. It is now an opaque type you can use instead of void.

Done, pushed the updated patch-set.

You also need to write man pages for these functions, see providers/mlx5/man/mlx5dv_devx_umem_reg.3.md for instance.

Sure, I will push this in the next revision if there are no other code related comments.

…

— Reply to this email directly, view it on GitHub <#1654 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTY2BIJHOVXBUWPLFMOGR34F7VWVAVCNFSM6AAAAACK7AEXQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMZQGA2DCMRWGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

sbasavapatna · 2026-01-15T10:44:10Z

While looking into changes needed in the bnxt_re kernel driver to process DBR passed through driver specific attributes (during QP creation), I noticed that the corresponding libibverbs support seems to be missing in cmd_qp.c.
That is, ibv_cmd_create_qp_ex() doesn't support "ibv_command_buffer *driver" arg that should be linked to the main command buffer that it creates. This is supported in CQ creation command though (ibv_cmd_create_cq_ex2()).

jgunthorpe · 2026-01-15T16:46:45Z

As the first user you'd need to do a similar thing as CQ.

sbasavapatna · 2026-01-17T08:16:37Z

On Thu, Jan 15, 2026 at 10:19 PM Jason Gunthorpe ***@***.***> wrote: *jgunthorpe* left a comment (linux-rdma/rdma-core#1654) <#1654 (comment)> As the first user you'd need to do a similar thing as CQ.

Pushed the next version with a new create_qp function in libibverbs. Thanks, -Harsha

…

— Reply to this email directly, view it on GitHub <#1654 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTY2BPOYJBYLUJ4JB5E2LL4G7AIXAVCNFSM6AAAAACK7AEXQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTONJVHAZTGNJUGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

jpirko · 2026-07-08T08:13:27Z

+	fill_attr_in_ptr(cmdb, attr_id, storage);
+	cmdb->fallback_ioctl_only = 1;
+}
+


Does not make sense to have this as separate helper. Your caller checks "buf->fd ==-1". You can do the check inside fill_attr_in_buf_umem and fill up things (like type) appropriately.

Done, will update it in the next version.

looks okay to me now.

jpirko · 2026-07-08T08:19:36Z

+};
+
+struct bnxt_re_dv_umem_reg_attr {
+	void *addr;


This duplicates the layout of struct ibv_buf. Why you can't use it directly?

This is a user facing structure and we don't want to change it as it would impact applications that are already using it.

This puzzles me. How an application can use this struct if you are only introducing it now?

OOB solution which already exists is being upstreamed.

Wait, you say that you form the API of a feature currently being pushed upon some out of tree code and use it as argument the API can't be changed? I have troubles to understand that reasoning, sorry.

The structure layouts are slightly different. We have added a comp_mask for vendor specific extensions.

struct bnxt_re_dv_umem_reg_attr {
void *addr;
size_t size;
uint64_t comp_mask;
int dmabuf_fd;
};
struct ibv_buf {
void *addr;
size_t size;
struct ibv_pd *pd;
int dmabuf_fd;
};

Also, since this is a Direct Verb to the user application, we want to maintain the same ABI for both our inbox and out-of-tree libraries, so that the applications can work seamlessly on both installations. So unless you see a design level issue here, we would prefer to keep this as-is and avoid any impact to existing customers that are using out-of-tree packages.

Why you have comp_mask here is a mystery. Not needed (and if really, really need it, you can embed ibv_buf - so no argument). Basically they are same structures. Don't screw-up the API in a name of out-of-tree code. That is fundamentally wrong. Please fix.

I agree with Jiri, we have ibv_buf now that is the user facing API surface for a dmabuf or a VA or whatever, you should add some API to allow the user to create an ibv_buf from other things and consistently use that in your here in DV. Get rid of the umem here, it completely duplicates what ibv_buf is trying to do.

You guys choose to do the OOT thing, the broken ABIs are all your problem to deal with.

Will add a couple of APIs as suggested to associate/disassociate an ibv_buf with user allocated memory. This ibv_buf will then be passed to bnxt_re_dv_create_cq/qp APIs.

jgunthorpe · 2026-08-07T18:41:47Z

+
+	ret = execute_ioctl(ibvctx, cmd);
+	if (ret) {
+		fprintf(stderr, "%s: execute_ioctl() failed: %d\n", __func__, ret);


Do not print errors from libraries like this, remove them all

will remove these error prints.

jgunthorpe · 2026-08-07T18:49:52Z

+				 (struct ibv_create_cq_ex *)&cmd.ibv_cmd,
+				 sizeof(cmd),
+				 (struct ib_uverbs_ex_create_cq_resp *)&resp.ibv_resp,
+				 sizeof(resp), 0))


Why all the casts here? It shouldn't have casts if written properly

will fix this.

jgunthorpe · 2026-08-07T18:55:11Z


 	if (ibv_cmd_create_qp_ex(ibvctx, &qp->vqp, attr,
-				&req.ibv_cmd, sizeof(req), &resp.ibv_resp, sizeof(resp)))
+				 &req.ibv_cmd, sizeof(req), &resp.ibv_resp, sizeof(resp)))


stray whitespace?

jgunthorpe · 2026-08-07T18:56:09Z

+				    (struct verbs_cq *)&cq->ibvcq,
+				    (struct ibv_create_cq_ex *)&cmd.ibv_cmd, sizeof(cmd),
+				    (struct ib_uverbs_ex_create_cq_resp *)&resp->ibv_resp,
+				    sizeof(*resp), 0, driver_attrs);


Also why casts?

will be fixed.

jgunthorpe · 2026-08-07T19:03:59Z

+};
+
+struct bnxt_re_dv_umem_reg_attr {
+	void *addr;


I agree with Jiri, we have ibv_buf now that is the user facing API surface for a dmabuf or a VA or whatever, you should add some API to allow the user to create an ibv_buf from other things and consistently use that in your here in DV. Get rid of the umem here, it completely duplicates what ibv_buf is trying to do.

You guys choose to do the OOT thing, the broken ABIs are all your problem to deal with.

jgunthorpe · 2026-08-07T19:06:05Z

+	dst->ah_attr.grh.flow_label = src->ah_attr.grh.flow_label;
+}
+
+int bnxt_re_dv_query_qp(struct ibv_qp *ibvqp, struct ib_uverbs_qp_attr *qp_attr)


This type ib_uverbs_qp_attr should not be returned from a library interface, it is a kernel facing type

I don't even understand why this function exists in the first place

Thanks for catching this; in the next version this will be restored to standard bnxt_re_query_qp() and the dv function will be removed.

jgunthorpe · 2026-08-07T19:06:55Z

+
+	rc = ibv_cmd_query_qp(ibvqp, &attr, qp_attr->qp_attr_mask, &init_attr,
+			      &cmd, sizeof(cmd));
+	if (!rc) {


success oriented flow

This function itself won't be there in the next version.

jgunthorpe · 2026-08-07T19:07:53Z

+registration, aligned to the device page size; **ncqe** is the requested number
+of CQEs.
+
+This path requires **IB_UVERBS_CORE_SUPPORT_ROBUST_UDATA** (robust udata) on the


Doesn't seem like something the user needs to know. You could write 'this call requires linux kernel version >= 'xyz

jgunthorpe · 2026-08-07T19:08:39Z

+```c
+#include <infiniband/bnxt_re_dv.h>
+
+int bnxt_re_dv_destroy_qp(struct ibv_qp *ibvqp);


Why does this exist? What is wrong with the normal QP destruction?

DV variants of QP functions - destroy, query and modify will be removed; they were mostly wrappers. Similarly dv_destroy_cq will also be removed.

mplaneta-qc · 2026-08-14T12:13:34Z

+struct bnxt_re_dv_db_region_attr {
+        uint32_t  handle;   /* kernel handle for this doorbell region */
+        uint32_t  dpi;      /* doorbell page index */
+        uint64_t  umdbr;    /* unmapped doorbell BAR offset */


What is the intended flow for using umdbr? Does unmapped mean not mapped into application virtual address space?

Yes, this is the raw PCI-BAR/physical-address of the doorbell register page.

How user space can use this field?

It is not meant for normal user-space applications. It is intended to support future extensions for peer devices like GPUs that may need to directly write to the Doorbell BAR.

Is it supposed to be used somehow with DMA buf then? https://www.kernel.org/doc/html/latest/driver-api/dma-buf.html

What? You cannot expose a physical address to userspace. Send a kernel patch to take this away please.

Can we pass a File Descriptor with help of dma-buf framework to user space instead of returning raw Physical address ?

What? You cannot expose a physical address to userspace. Send a kernel patch to take this away please.

Since the ABI structure is already in released kernels (7.1, 7.2), we can't remove the field right? I have a patch that changes it to a reserved field and avoids setting it. If that's ok, I can send out that patch, let me know.

To commit: 23d7e03a52ec ("RDMA/bnxt_re: Avoid exposing umdbr to userspace"). Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>

fill_attr_in_buf_umem() only handles dmabuf-backed buffers. Extend it for user virtual address buffers. Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>

ibv_alloc_buf() only allocates library memory and initializes an ibv_buf for it; there's no way to associate an ibv_buf with memory the application already owns. Add ibv_alloc_user_buf()/ibv_free_user_buf() so providers can do that (VA or dmabuf) for direct consumption by driver-specific control-buffer (e.g queue memory) descriptors. Applications will call this to associate their own CQ/QP buffers as an ibv_buf. The resulting ibv_buf is then passed to the CQ/QP creation Direct Verb APIs added later in this series. Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>

Extract HSI definitions (WQE/CQE structures and enums) from bnxt_re-abi.h into a standalone bnxt_re-hsi.h and add it to publish_headers() so it is installed under infiniband/. This will make HSI definitions available to DV applications, that implement their own datapath. Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com> Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>

The following Direct Verb (DV) APIs have been implemented in this patch. Doorbell Region Direct Verbs: - bnxt_re_dv_alloc_db_region(): This will allow the appliation to create extra doorbell regions and use the associated doorbell page index in dv_create_qp() and use the associated DB address while ringing the doorbell. - bnxt_re_dv_free_db_region(): Free the allocated doorbell region. - bnxt_re_dv_get_default_db_region(): Return the default doorbell page index and doorbell page address associated with the ucontext. Co-developed-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com> Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com> Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com> Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>

The next patch in this series switches to ibv_cmd_create_cq_ex()/_ex2(), which take a struct verbs_cq as an argument instead of ibv_cq. So update bnxt_re_cq to replace ibv_cq with verbs_cq. Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>

The following Direct Verb (DV) API has been implemented in this patch, by enhancing the driver specific udata in the existing verb. - bnxt_re_dv_create_cq(): Create a CQ of requested size (cqe). The application must have already associated this memory with an ibv_buf using ibv_alloc_user_buf(). The CQ's umem_handle and umem_offset provided by the application are translated into an address for mapping and passed to the driver. Note: Some applications might want to allocate memory for all CQs in one big chunk and associate that entire memory with a single ibv_buf using ibv_alloc_user_buf(). At the time of creating each individual resource, the application should pass a specific offset/length within that memory. Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com> Co-developed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com> Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com> Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>

The following Direct Verb (DV) API has been implemented in this patch, by enhancing the driver specific udata in the existing verb. - bnxt_re_dv_create_qp(): Create a QP using specified params (struct bnxt_re_dv_qp_init_attr). The application must have already associated SQ/RQ memory with an ibv_buf using ibv_alloc_user_buf(). The SQ/RQ umem_handle and umem_offset provided by the application are translated into an address for mapping and passed to the driver. Note: Some applications might want to allocate memory for all QPs in one big chunk and associate that entire memory with a single ibv_buf using ibv_alloc_user_buf(). At the time of creating each individual resource, the application should pass a specific offset/length within that memory. Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com> Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>

Document direct verbs (DV) feature and related DV APIs. The man page describes these DV APIs: - Doorbell regions (db) - Completion Queues (cq) - Queue pairs (qp) Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com> Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>

sbasavapatna force-pushed the dv-upstream branch from 8042660 to 2617428 Compare November 10, 2025 08:02

sbasavapatna force-pushed the dv-upstream branch from 2617428 to 7c19ba0 Compare November 18, 2025 16:12

sbasavapatna force-pushed the dv-upstream branch from 7c19ba0 to 130c041 Compare November 29, 2025 17:04

jgunthorpe reviewed Dec 17, 2025

View reviewed changes

sbasavapatna force-pushed the dv-upstream branch from 130c041 to c841ce1 Compare December 24, 2025 04:19

sbasavapatna force-pushed the dv-upstream branch from c841ce1 to 37a3df4 Compare January 13, 2026 17:20

sbasavapatna force-pushed the dv-upstream branch from 37a3df4 to d6d9097 Compare January 17, 2026 08:12

sbasavapatna force-pushed the dv-upstream branch 2 times, most recently from 294a076 to 3d8c351 Compare February 3, 2026 05:11

sbasavapatna force-pushed the dv-upstream branch 2 times, most recently from d43117e to 362804c Compare February 11, 2026 12:59

sbasavapatna closed this Apr 7, 2026

sbasavapatna force-pushed the dv-upstream branch from 362804c to cc0e95c Compare April 7, 2026 06:27

sbasavapatna reopened this Apr 7, 2026

sbasavapatna force-pushed the dv-upstream branch 3 times, most recently from 7a10dd5 to 24a1e0f Compare April 8, 2026 08:33

sbasavapatna requested a review from jgunthorpe April 8, 2026 08:34

sbasavapatna closed this Jun 17, 2026

sbasavapatna reopened this Jun 17, 2026

sbasavapatna force-pushed the dv-upstream branch 4 times, most recently from 530f887 to 28d4255 Compare June 17, 2026 10:16

sbasavapatna closed this Jul 3, 2026

sbasavapatna force-pushed the dv-upstream branch from 28d4255 to 9143bdd Compare July 3, 2026 07:38

sbasavapatna reopened this Jul 3, 2026

jpirko reviewed Jul 8, 2026

View reviewed changes

sbasavapatna force-pushed the dv-upstream branch from 66be44e to 694f4b4 Compare July 8, 2026 10:19

sbasavapatna force-pushed the dv-upstream branch from 694f4b4 to 5fddf2f Compare July 16, 2026 09:24

jgunthorpe requested changes Aug 7, 2026

View reviewed changes

mplaneta-qc reviewed Aug 14, 2026

View reviewed changes

sbasavapatna closed this Aug 21, 2026

sbasavapatna force-pushed the dv-upstream branch from 1b04a74 to 691953d Compare August 21, 2026 16:22

sbasavapatna reopened this Aug 21, 2026

sbasavapatna force-pushed the dv-upstream branch 2 times, most recently from 35fcb9e to 1732e14 Compare August 24, 2026 18:01

sbasavapatna and others added 10 commits September 9, 2026 11:27

Merge branch 'master' into dv-upstream

94813ea

Update kernel headers

ccb88ab

To commit: 23d7e03a52ec ("RDMA/bnxt_re: Avoid exposing umdbr to userspace"). Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>

verbs: Extend fill_attr_in_buf_umem() for VA-backed descriptors

c4fbda3

fill_attr_in_buf_umem() only handles dmabuf-backed buffers. Extend it for user virtual address buffers. Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>

sbasavapatna force-pushed the dv-upstream branch from 68e693f to 94813ea Compare September 9, 2026 06:13

Uh oh!

Conversation

sbasavapatna commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jgunthorpe commented Dec 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbasavapatna commented Dec 24, 2025 via email

Uh oh!

sbasavapatna commented Dec 24, 2025 via email

Uh oh!

jgunthorpe commented Jan 7, 2026

Uh oh!

sbasavapatna commented Jan 8, 2026 via email

Uh oh!

sbasavapatna commented Jan 8, 2026

Uh oh!

jgunthorpe commented Jan 9, 2026

Uh oh!

sbasavapatna commented Jan 13, 2026 via email

Uh oh!

sbasavapatna commented Jan 15, 2026

Uh oh!

jgunthorpe commented Jan 15, 2026

Uh oh!

sbasavapatna commented Jan 17, 2026 via email

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

sbasavapatna commented Nov 3, 2025 •

edited

Loading

Aswin5005 Aug 24, 2026 •

edited

Loading