Blab AI
Columbia Study Shows Older Nvidia A40 GPUs in Paraguay Match Newer H100s for LLM Pre-Training
A recent study by researchers from Columbia University’s Department of Industrial Engineering and Operations Research has shown that Nvidia A40 graphics processors located in Asuncion, Paraguay, can deliver performance comparable to the newer Nvidia H100 chips when training large‑language models (LLMs). The work, submitted to the Neural Information Processing Systems (NeurIPS) conference, focused on pre‑training models with up to 1.4 billion parameters.
The research was conducted in the context of HIVE Digital Technologies’ first AI GPU cluster, which went live in March 2026. The cluster, powered by hydroelectric energy from the Itaipú Dam, uses a fleet of A40 GPUs. Columbia researchers operated the training jobs from New York, more than 5,000 miles away, demonstrating an intercontinental AI training workflow.
According to the study, the team spent two months optimizing code for the A40 processors. After normalizing for each platform’s raw hardware capabilities, the performance achieved on the A40 systems matched that observed on Nvidia’s flagship H100 processors. A researcher from Columbia noted, “In our use case of pretraining LLMs of up to 1.4 B parameters, our results match those of H100s after normalizing for each hardware’s raw performance.”
The findings underscore the importance of software optimization in AI computing. They also suggest that developers can access compute resources globally without significant performance penalties, potentially broadening the range of locations suitable for advanced AI infrastructure. The work aligns with HIVE’s plan to build a 100‑MW AI and high‑performance computing campus in Paraguay.
HIVE’s executive chairman and co‑founder, Frank Holmes, said the proof‑of‑concept is an important step in the company’s mission to bring advanced AI computing infrastructure to Paraguay. The study’s submission to NeurIPS signals the research community’s interest in remote, renewable‑powered AI training.
In summary, the research confirms that older GPUs can compete with newer models when software is carefully tuned. It also validates HIVE’s strategy of leveraging low‑cost, renewable‑powered sites for large‑scale AI workloads. The Asuncion cluster remains the first operational GPU cluster under HIVE’s phased approach to layer AI and high‑performance computing onto its existing renewable energy infrastructure, and the company plans to expand its HPC footprint in the region.
The research was conducted in the context of HIVE Digital Technologies’ first AI GPU cluster, which went live in March 2026. The cluster, powered by hydroelectric energy from the Itaipú Dam, uses a fleet of A40 GPUs. Columbia researchers operated the training jobs from New York, more than 5,000 miles away, demonstrating an intercontinental AI training workflow.
According to the study, the team spent two months optimizing code for the A40 processors. After normalizing for each platform’s raw hardware capabilities, the performance achieved on the A40 systems matched that observed on Nvidia’s flagship H100 processors. A researcher from Columbia noted, “In our use case of pretraining LLMs of up to 1.4 B parameters, our results match those of H100s after normalizing for each hardware’s raw performance.”
The findings underscore the importance of software optimization in AI computing. They also suggest that developers can access compute resources globally without significant performance penalties, potentially broadening the range of locations suitable for advanced AI infrastructure. The work aligns with HIVE’s plan to build a 100‑MW AI and high‑performance computing campus in Paraguay.
HIVE’s executive chairman and co‑founder, Frank Holmes, said the proof‑of‑concept is an important step in the company’s mission to bring advanced AI computing infrastructure to Paraguay. The study’s submission to NeurIPS signals the research community’s interest in remote, renewable‑powered AI training.
In summary, the research confirms that older GPUs can compete with newer models when software is carefully tuned. It also validates HIVE’s strategy of leveraging low‑cost, renewable‑powered sites for large‑scale AI workloads. The Asuncion cluster remains the first operational GPU cluster under HIVE’s phased approach to layer AI and high‑performance computing onto its existing renewable energy infrastructure, and the company plans to expand its HPC footprint in the region.
