Octo Metric, LLC

DOCUMENT FOR PUBLIC RELEASE

The decision issued on the date below was subject to a GAO Protective Order. The entire decision has now been approved for public release.

Decision

Matter of: Octo Metric, LLC

File: B-424623

Date: October 8, 2026

Damien C. Specht, Esq., James A. Tucker, Esq., and Thomas Lee, Esq., Morrison & Foerster LLP, for the protester.

Alexander Gorelik, Esq., C. Kelly Kroll, Esq., and Kelly A. Carlson, Esq., Taft Stettinius & Hollister, LLP, for AEX Digital, LLC, the intervenor.

Richard W. Postma, Esq., Department of Homeland Security, for the agency.

Glenn G. Wolcott, Esq., and April Y. Shields, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Protest is denied where the agency reasonably evaluated the protester's proposed solution in a manner that was consistent with the solicitation's evaluation criteria.

DECISION

Octo Metric LLC (hereinafter “Octo”), of Atlanta, Georgia, protests the issuance of a task order by the Department of Homeland Security, U.S. Citizenship and Immigration Services (USCIS), to AEX Digital, LLC, of Bethesda, Maryland, pursuant to request for quotations (RFQ) No. 70SBUR26Q00000052, to provide information technology (IT) support services. Octo, the incumbent contractor, asserts that the agency's evaluation of its quotation was unreasonable. More specifically, Octo asserts that the evaluation was “prejudicially undermined by three weaknesses unreasonably assessed.” Protest at 1, 5-12.[1]

We deny the protest.

BACKGROUND

On March 16, 2026, pursuant to subpart 8.4 of the Federal Acquisition Regulation (FAR),[2] the agency issued the RFQ as a total small business set-aside to vendors holding General Services Administration multiple award schedule 54151S contracts.[3] Agency Report (AR), Tab 1a, RFQ at 68. As amended, the solicitation contemplated the issuance of a single fixed-price task order to provide IT services supporting USCIS's mission.[4] More specifically, the solicitation sought proposals to “support the design, architecture, configuration, development, and maintenance of the current technological solution while enhancing USCIS's analytical capabilities.”[5] See AR, Tab 5, PWS at 4. The RFQ further provided for a best-value source selection process based on a tradeoff between price and the following non-price evaluation factors, listed in descending order of importance: (1) code challenge; and (2) corporate experience.[6] AR, Tab 10, RFQ amend. 0001 at 59-64.

In this context, the solicitation divided the code challenge requirements[7] into two parts: (1) a take-home portion;[8] and (2) a live demonstration/implementation of the solution and new requirements.[9] Id. at 55. The second portion was divided into time periods during which various activities were to be performed, including a three-hour period during which the vendor “will implement the new requirements.”[10] Id. at 56-57.

Finally, the solicitation identified various criteria the agency would consider in its evaluation, including assessment of: whether “the take-home portion and the new challenge requirements solution[] are deployed and presented in a production environment”; the vendor's “product and program management practices, as well as the methodology”; and whether the vendor's solution “incorporate[s] innovative, practical, cost-effective platform methods and techniques that align with the Agency's processes and quality standards.” Id. at 60-61.

On March 31, quotations were submitted by six vendors, including AEX and Octo; thereafter, AEX's and Octo's quotations were evaluated as follows:[11]

Code Challenge

Corporate Experience

Evaluated Price

AEX

High Confidence

Some Confidence

$88,090,259

Octo

Some Confidence

High Confidence

$100,598,462

AR, Tab 32, Selection Decision Document (SDD) at 6.

In assigning a rating of some confidence to Octo's quotation under the most important factor, code challenge, the agency identified three aspects of the quotation that decreased the agency's confidence.[12] First, the agency determined that “not all code challenge additional requirements were deployed and presented in a production environment.” AR, Tab 30, TEC Report at 19. Next, the agency found that Octo “did not demonstrate strong program management practices or a sufficient methodology for developing and implementing its proposed solution.”[13] Id. Finally, the agency concluded that Octo's solution “lacked practical, cost-effective platform methods because [Octo's] cost tracking dashboard demonstrated that the solution resulted in high operational costs.” Id.

Based on the evaluation summarized above, the agency concluded that AEX's quotation, as the highest-technically-rated and lowest-priced quotation, reflected the best value for the government; accordingly, a task order was issued to AEX on June 22. This protest followed.

DISCUSSION

Octo challenges each of the three aspects of its quotation that the agency identified as decreasing its confidence in Octo's solution. As noted above, these were: (1) failure to deploy and present all of the additional requirements in a production environment; (2) failure to demonstrate strong program management practices or sufficient methodology for developing and implementing its proposed solution; and (3) failure to propose a cost‑effective platform. As discussed below, we find no merit in any of Octo's complaints.

Failure to Deploy and Present Additional Requirements in a Production Environment

In its protest, Octo initially asserts that during the code challenge it “presented all of the analytical and data engineering capabilities that constituted the heart of the additional requirements.” Protest at 7. Following its receipt and review of the agency report responding to the protest, Octo states that it “deployed a majority of [the new] requirements.” Comments at 3 n.1. Further, Octo asserts that “to the extent [Octo] failed to deploy its solution [its] failure to do so was the result of circumstances beyond [Octo's] control.” Id. at 2; see also Protest at 8. More specifically, Octo's solution used a software tool called Terraform, and Octo complains that there was “an issue with a Terraform software update that rendered its underlying technical solution unworkable without time-consuming workarounds.”[14] Comments at 2, 3 n.1. In addition, Octo complains that “Databricks issued an update to its platform that affected how Databricks would download and verify . . . Terraform.” Protest at 8. In short, Octo maintains that, due to the “upgrade issues with Databricks and Terraform,” the agency “unreasonably failed to consider” that these updates “contributed to some of [Octo' s] undeployed elements of its solution.” Comments at 5.

The agency first responds by pointing out what Octo has conceded: that Octo, in fact, failed to deploy all of the new requirements to a production environment, as required by the solicitation. See AR, Tab 10, RFQ amend. 0001 at 60. For example, the agency points out that following the three‑hour period for implementation of the new requirements, the agency asked Octo: “[D]id all of the new requirements deploy[] to production?” AR, Tab 24, Transcript Part 2 at 44. In response, Octo initially asserted: “I think all new requirements [were] deployed. I think maybe.” Id. However, after some discussion, Octo acknowledged that: “We had to quickly update the bundles. If we just had like one thing we missed, I could fix it in 10 seconds with the right app. But we didn't deploy anything to prod[uction].” Id.

Next, the agency responds to Octo's assertion that it should not have been criticized for its deployment failure because the failure was caused by software updates that Octo maintains were “beyond [its] control.” See Comments at 2. In this regard, the agency first notes that, while all vendors were required to use Databricks, Octo chose to rely on the Terraform software as part of its solution; accordingly, it was responsible for staying apprised of changes that could affect its chosen solution. AR, Tab 35, TEC Chair Statement at 21. Next, the agency responds that the Databricks update “was not an issue unique to [Octo]” and, accordingly, the agency “had the opportunity to observe and evaluate how [vendors] would individually respond . . . and how efficiently they could resolve such issues.” Id. In this context, the agency notes that Octo “spent a significant portion of its three-hour coding window (over two hours . . .) trying to resolve this issue.”[15] Id. The agency further determined that Octo “did not have personnel with the necessary skills to resolve the upgrade issue . . . in a timely manner,” and it was Octo's “inability to resolve the issue” that “prevented . . . them from deploying.” Id. In short, the agency maintains that its evaluation of Octo with regard to this matter was reasonable and consistent with the solicitation requirements. We agree with the agency.

The evaluation of quotations is a matter within the discretion of the procuring agency. Highlight Techs., Inc., B-423837.3, Feb. 19, 2026, at 4-5. The agency is responsible for defining its needs and the best method of accommodating them. Id. Where a protester challenges the agency's evaluation in a competition for a task order under FAR subpart 8.4, our Office will review the record to ensure that the agency's evaluation is reasonable and consistent with the terms of the solicitation. See id. A protester's disagreement with the agency's judgment, by itself, is insufficient to establish that an agency acted unreasonably. Id.

Here, based on the record presented and for the reasons discussed by the agency in responding to Octo's protest, we find no basis to question the agency's determination that, during its live demonstration, Octo failed to deploy all of the new code challenge requirements to the production environment as contemplated by the solicitation. Similarly, we reject Octo's assertion that the agency's criticism of this failure was improper due to matters “beyond [Octo's] control.” First, with regard to the update to the Terraform software, it was Octo's decision to incorporate this software tool into its solution; accordingly, we find no basis to question the agency's conclusion that Octo was responsible for staying apprised of changes that could affect a portion of its chosen solution. We also find no basis to question the agency's assessment of Octo's capability to deal with updates to Databricks (which the solicitation required all vendors to incorporate into their solutions) given the length of time Octo took to address that issue. In short, Octo has provided no meaningful basis to challenge the agency's judgment that Octo's inability to resolve the issues in a timely manner formed a reasonable basis for the agency's assessment of “decreased confidence” in Octo's solution. Accordingly, Octo's protest challenging this aspect of the agency's evaluation is denied.

Program Management and Methodology

Next, Octo challenges the agency's identification of Octo's program management and methodology as a basis for decreased confidence in Octo's solution.[16] Protest at 9-10. Octo disputes the agency's assessment, asserting that its personnel are “all highly trained . . . with extensive experience successfully performing the requirements under the incumbent task order,” and refers to what it describes as Octo's “nimble diagnosis and remediation of the serious problem caused by Databricks.” Id. at 9. In short, Octo asserts that the agency's second assessment of “decreased confidence” was unreasonable. Comments at 7.

In response, the agency first notes that the solicitation did not provide for evaluation of Octo's incumbency. AR, Tab 35, TEC Chair Statement at 24. Next, the agency notes that, rather than a “nimble diagnosis and remediation” of the problems discussed above, the interaction between Octo's team reflected a “lack of clear direction,” “gaps in leadership,” and “deficiencies in managing and communicating project outcomes.” Id. at 25. For example, the agency points to Octo's discussion in responding to the agency's questions regarding deployment of the new requirements, describing that discussion as “uncertain, scattershot,” and “otherwise equivocal.” MOL at 14; see AR, Tab 24, Transcript Part 2 at 44-51. By way of another example, the agency notes that Octo's solution architect (who was identified as critical to Octo's solution) “was unable to effectively lead the team or demonstrate the decisiveness and adaptability needed to address bottlenecks and improve outcomes,” and presents various screenshots from the Microsoft Teams recording, along with dialogue in the transcript, that support the agency's assessment. AR, Tab 35, TEC Chair Statement at 25-31; see AR, Tab 23, Transcript Part 1 at 56; AR, Tab 24, Transcript Part 2 at 3, 21, 43, 48‑49. Finally, the agency points out, as noted above, that Octo itself acknowledged that it “wasted” two of the three hours allocated to implementing the new requirements. See AR, Tab 24, Transcript Part 2 at 3. In short, the agency maintains that it reasonably identified Octo's program management and methodology as a basis for its decreased confidence in Octo's solution.

As noted above, our Office will not reevaluate quotations but, rather, will review the record to ensure that the agency's evaluation is reasonable and consistent with the terms of the solicitation; a protester's disagreement, by itself, is insufficient to establish that an agency acted unreasonably. Highlight Techs., Inc., supra.

Here, based on our review of the record, we find no basis to question the agency's evaluation with regard to Octo's program management and methodology. As noted above, the solicitation provided for the agency to assess the vendor's “product and program management practices” and, in that context, the agency criticized Octo for “limited teaming, collaboration and gaps in product and program management practices.” AR, Tab 10, RFQ amend. 0001 at 61; AR, Tab 30, TEC Report at 19. Consistent with the agency's criticism, the record establishes that, in responding to the agency's question regarding deployment of new requirements, Octo's team vacillated between asserting that they believed everything had been deployed to acknowledging that it had not. See AR, Tab 24, Transcript Part 2 at 44. Similarly, based on our review of the agency's references to various portions of the recording and transcripts documenting the interactions between Octo's team during the code challenge, see AR, Tab 35, TEC Chair Statement at 25-31, we find no basis to question the agency's stated concerns. Finally, Octo itself acknowledges that it “wasted” two-thirds of the time allotted to implementation of the new requirements. On this record, in our view, the agency reasonably viewed Octo's program management and methodology as a basis for the agency's decreased confidence, and Octo's protest merely reflects its disagreement with the agency's judgments in that regard. Accordingly, Octo's protest challenging this aspect of the evaluation is denied.

Cost-Effective Methods and Techniques

Finally, Octo complains that the agency improperly identified Octo's “high operational costs” as an aspect of Octo's solution that decreased the agency's confidence.[17] Protest at 11-12. In this context, Octo identifies various aspects of its proposed solution that it believes responded to the agency's stated objectives--including “near real-time, highly accurate, person-centric data sharing”--and asserts that such capabilities “will necessarily carry higher infrastructure costs.” Id. Accordingly, Octo maintains that the agency's negative assessment was improper.

The agency responds that its assessment regarding the cost-effectiveness of Octo's solution was based on the cost information that Octo displayed during its demonstration and, based on that information, the agency concluded that Octo “made the decision to continuously run the pipeline, which increased its costs.” COS at 8. In this context, the agency elaborates that Octo's “design choices” prioritized “persistent platform operation over fiscal discipline,” and failed to implement “cost-control mechanisms.” AR, Tab 35, TEC Chair Statement at 33-34. In short, the agency states that Octo's solution “relied on continuously running jobs and did not demonstrate clear mechanisms to minimize idle or unnecessary resource consumption.” Id. at 35. Accordingly, the agency maintains that its assessment that Octo's proposed solution was not cost-effective was reasonable and consistent with the terms of the solicitation.

As noted above, our Office will not reevaluate quotations but, rather, will review the record to ensure that the agency's evaluation is reasonable and consistent with the terms of the solicitation; a protester's disagreement, by itself, is insufficient to establish that an agency acted unreasonably. Highlight Techs., Inc., supra.

Here, based on our review of the record, we find no basis to question the agency's evaluation regarding the cost-effectiveness of Octo's proposed solution. As noted above, the solicitation specifically directed vendors to “incorporate innovative, practical, cost-effective platform methods and techniques that align with the Agency's processes and quality standards.” See AR, Tab 10, RFQ amend. 0001 at 60. Further, Octo does not dispute that its proposed approach will “necessarily carry higher infrastructure costs.” See Protest at 11. On this record, we again view Octo's complaints regarding the agency's assessment as reflecting Octo's disagreement with the agency's judgment, and Octo has failed to demonstrate that the agency's judgment in this regard was unreasonable. Accordingly, Octo's complaints regarding the agency's judgment in this matter are denied.[18]

The protest is denied.

Edda Emmanuelli Perez

General Counsel

[1] The page numbers referenced in this decision are PDF page numbers for the various pleadings and documents submitted.

[2] The RFQ and the agency's filings reference both the FAR and the Revolutionary FAR Overhaul (RFO). See Contracting Officer's Statement (COS) at 1; Memorandum of Law (MOL) at 1, 4; AR, Tab 10, RFQ amend. 0001 at 3. The parties have not established that any distinctions between the prior and the RFO versions of FAR part 8 affect the analysis of the agency's actions in this procurement.

[3] This procurement is referred to as “Predictive Lake Analytics Nextgen eXchange Services II” or “PLANXS II.” COS at 1.

[4] Under the heading “USCIS Mission,” the solicitation states that USCIS “administers the nation's lawful immigration system, safeguarding its integrity and promise by efficiently and fairly adjudicating requests for immigration benefits.” AR, Tab 5, Performance Work Statement (PWS) at 3.

[5] The RFQ described the various IT systems that make up the current technological solution as including an “Enterprise Data Lake,” which “[u]ses Databricks . . . and enables data scientists to perform advanced analytics using tools to extract value and reveal insights from datasets that are diverse, complex, and of a massive scale.” AR, Tab 5, PWS at 4. The protester describes “Databricks” as “a widely used, cloud-based software platform that helps organizations store, process, and analyze very large amounts of data, and build artificial intelligence applications on top of that data.” Protest at 2 n.2. Among other things, the solicitation provided that vendors' solutions must “us[e] Databricks as a platform to unify and consolidate data.” AR, Tab 10 RFQ amend. 0001 at 55.

[6] The solicitation stated that the non-price factors, when combined, “are significantly more importan[t] than price.” AR, Tab 10, RFQ amend. 0001 at 47.

[7] The agency explained that “[t]he objective of this challenge is to develop a deployable solution that proactively identifies predictive patterns and anomalies in data, focusing on key fields from official USCIS form numbers and instructions provided in the artifacts.” AR, Tab 37, Code Challenge Instructions at 1.

[8] The solicitation provided that vendors “will be given fourteen (14) calendar days to implement a solution to [an agency-provided] problem statement and submit” their solution. AR, Tab 10, RFQ amend. 0001 at 55.

[9] More specifically, the solicitation provided that during the live demonstration, the vendor “shall be asked by the Government to implement enhancements or changes to [its] solution,” noting that each vendor “will be given the same new requirement.” Id. at 55, 57.

[10] The solicitation further provided that the live portion of the code challenge would be conducted “via Microsoft Teams and all presentations will be recorded with transcriptions enabled.” Id. at 57.

[11] In evaluating quotations under the non-price factors, the RFQ provided that the agency would make assessments of high confidence, some confidence, or low confidence regarding the vendors' understanding of the requirements, proposed approach, and potential for successfully performing the task order. Id. at 64. In this context, the agency identified various aspects of each vendor's quotation that “increased or decreased the Government's confidence” in the vendor's solution. AR, Tab 30, Technical Evaluation Committee (TEC) Report at 8.

[12] The agency also identified two aspects of Octo's quotation that increased the agency's confidence. AR, Tab 30, TEC Report at 18-19.

[13] In this regard, the agency added that “[t]his resulted in limited teaming, collaboration and gaps in product and program management practices, impacting on the team's ability to meet the project and product management requirements outlined [in the solicitation].” Id. at 19.

[14] Although the solicitation required vendors to use Databricks, it did not require use of Terraform software as part of vendors' solutions. See AR, Tab 35, TEC Chair Statement at 21-22; AR, Tab 10 RFQ amend. 0001 at 55.

[15] Octo itself expressly acknowledged during the live demonstration that it “wasted 2 hours on a [Databricks] update.” AR, Tab 24, Transcript Part 2 at 3.

[16] As referenced above, the agency made the following assessment: [Octo] did not demonstrate strong program management practices or a sufficient methodology for developing and implementing its proposed solution. This resulted in limited teaming, collaboration and gaps in product and program management practices, impacting on the team's ability to meet the project and product management requirements outlined [in the solicitation]. AR, Tab 30, TEC Report at 19.

[17] As noted above, the solicitation provided that the agency would assess whether the vendor's solution “incorporate[d] . . . cost-effective platform methods and techniques that align with the Agency's processes and quality standards.” AR, Tab 10, RFQ amend. 0001 at 60. In its evaluation, the agency concluded that Octo's solution “lacked practical, cost-effective platform methods because [Octo's] cost tracking dashboard demonstrated that the solution resulted in high operational costs.” AR, Tab 30, TEC Report at 19.

[18] Octo also asserts that the agency's best-value tradeoff was “unreasonable” based on the agency's “underlying evaluation errors.” Protest at 12. Since this allegation is based on the protester's various challenges to the agency's evaluation which we have rejected, Octo's challenge to the best-value determination does not provide an independent basis of protest. See, e.g., Saalex Sols., Inc., B-418729.3, July 23, 2021, at 7.