Repository navigation
[Livy-1091] Upgrade libthrift 0.9.3 to 0.25.0 - #567
nileshrathi345 wants to merge 2 commits into
Conversation
7509144 to
3c6eb8f
Compare
|
Hi @nileshrathi345 , Thanks for the PR! Unfortunately there is a CI failure, could you please fix it? Also, please double check locally if all unit and integration tests run without errors for both Spark 4 and Spark 3 using the following commands: |
Hive JDBC 3.0 (used by Thrift HTTP integration tests) still constructs THttpClient with Apache HttpClient 4.x: THttpClient(String, org.apache.http.client.HttpClient) libthrift 0.25 only provides HttpClient 5.x constructors, which causes NoSuchMethodError in HttpThriftServerSuite at runtime. Add a test-scoped THttpClient under src/test/java that restores the HttpClient 4.x constructor for the JDBC test client. The Livy Thrift HTTP server uses ThriftHttpServlet (not libthrift TServlet/THttpClient), so this does not change production server artifacts.
Hi @roczei — thanks for the review. Why CI failed After moving to libthrift 0.25, HttpThriftServerSuite failed with: NoSuchMethodError: THttpClient.(String, org.apache.http.client.HttpClient) The HTTP thrift integration tests connect with Hive JDBC 3.0, which still builds the HTTP transport using Apache HttpClient 4.x and that constructor. libthrift 0.25 still has THttpClient, but it only supports HttpClient 5.x — the HttpClient 4 constructor was removed. So the class is present, but the method Hive JDBC calls is not → runtime failure when the suite opens a JDBC connection in HTTP mode. This is on the test client (JDBC driver) side. The Livy Thrift HTTP server uses our ThriftHttpServlet path, not libthrift’s THttpClient. Fix: Added a test-only org.apache.thrift.transport.THttpClient under thriftserver/server/src/test/java that restores the HttpClient 4.x constructor for the JDBC client during ITs. It is not part of the shipped server artifact. Local verification Both commands completed successfully on my side: mvn -Pthriftserver -Pspark3 -Pscala-2.12 -B verify nilesh.rathi@HQV9WP4F7Y livy % mvn -Pthriftserver -B verify |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #567 +/- ##
============================================
+ Coverage 69.03% 69.13% +0.09%
- Complexity 1231 1232 +1
============================================
Files 107 107
Lines 6986 6979 -7
Branches 858 857 -1
============================================
+ Hits 4823 4825 +2
+ Misses 1689 1682 -7
+ Partials 474 472 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
3 participants
What changes were proposed in this pull request?
This PR upgrades the Livy Thrift Server dependency on org.apache.thrift:libthrift from 0.9.3 to 0.25.0 (LIVY-1091).
Motivation: Apache Thrift before 0.23.0 is affected by CVE-2026-43869 (TLS hostname verification in TSSLTransportFactory). Livy’s Thrift Server uses libthrift for binary and HTTP CLI/JDBC transport; bumping to 0.25.0 addresses that class of issues and removes a very old pinned dependency.
Jira: https://issues.apache.org/jira/browse/LIVY-1091
How was this patch tested?
Compile (Spark 3):
mvn -Pthriftserver -Pspark3 -Pscala-2.12 -pl thriftserver/server,thriftserver/session -am compile -DskipTestsCompile (Spark 4 / default):
mvn -Pthriftserver -pl thriftserver/server,thriftserver/session -am compile -DskipTestsUnit tests:
mvn -Pthriftserver -Pspark3 -Pscala-2.12 -pl thriftserver/session test(ColumnBufferTest, 6 tests)
Was this patch authored or co-authored using generative AI tooling?
Co-authored by: Cursor AI