Why Financial Institutions Choose “On-Premises AI” Over the Cloud
VIEW CARD, a credit card company, has introduced generative AI for its debt management operations. What’s noteworthy is the deployment method. Instead of a cloud-based solution, they adopted “Court,” an on-premises system for drafting legal pleadings that runs on their own servers. This choice indicates that AI adoption in the financial industry is moving beyond mere efficiency gains.
Debt management, especially the creation of litigation-related documents, requires high accuracy and confidentiality. Transmitting documents containing customer information, transaction history, and legal arguments to an external cloud service carries significant risk. The dilemma between the convenience of generative AI and the data privacy that financial institutions must protect is resolved by adopting on-premises AI.
Running AI models within a company’s own firewall dramatically reduces the risk of data leakage. This addresses the primary concern of many hesitant executives, particularly in finance, healthcare, and legal sectors. The case of Ippu Senkin and VIEW CARD presents a practical solution: “leveraging AI while ensuring security.”
Diverging Strategies: “ChatSense” Transcription vs. On-Premises
Another notable development is the release of the transcription and email sharing feature in “ChatSense,” an AI platform for large enterprises, offered as a cloud-based service. Comparing these two cases reveals a clear “strategic fork in the road” for AI implementation.
ChatSense addresses the “visualization and sharing of information” from meeting minutes and customer emails. It suits tasks with relatively lower confidentiality requirements, prioritizing speed and collaboration. In contrast, VIEW CARD’s on-premises AI aims for “automating highly confidential, routine tasks.”
When executives or CTOs consider implementing AI, this is the first question they should ask: “Which tasks should be automated, and in which environment?” It’s not a binary choice between outsourcing everything to the cloud or building everything in-house. A hybrid approach, selecting the optimal environment based on task characteristics, is the practical answer.
The Concrete Costs and Hurdles of On-Premises AI Implementation
So, what are the realistic costs and technical capabilities required to build and implement an on-premises generative AI system? This is a practical concern for many executives.
First, there are hardware costs. Running the latest generative AI models smoothly requires servers equipped with high-performance GPUs. For example, introducing a single NVIDIA A100 or H100 involves an initial investment of several million yen. However, if the task is limited to generating specific, routine documents like “pleading drafts,” it’s possible to operate with smaller models or less powerful GPUs, potentially reducing the initial investment to around several hundred thousand yen.
Next are software and operational costs. This involves downloading open-source generative AI models (like Llama 3 or Mistral) onto company servers and fine-tuning them for specific business tasks. This process requires personnel with expertise in AI engineering or MLOps. If such talent isn’t available in-house, outsourcing to a systems integrator is common, with development costs also reaching the million-yen range.
However, a significant shift is emerging. The evolution of code-generating AI (like Claude Code, GitHub Copilot) is lowering the barrier to building and maintaining AI systems. We are entering an era where employees with a certain level of IT literacy can use these tools to create system frameworks, even without a full-stack engineer on staff. This means small and medium-sized enterprises can also realistically consider on-premises AI as an option.
The New Model Opened by “AI Co-Creation” Between Government and Business
Another piece of news is that SDT Corporation in Fujisawa City has signed a collaboration agreement with Namerikawa City in Toyama Prefecture for generative AI utilization. This demonstrates a new form of “AI co-creation,” combining the administrative challenges of local governments with the AI expertise of tech companies.
The interesting aspect of this collaboration is the mutual complementarity of resources. Local governments have a clear mission to improve resident services, along with the budget and (anonymized) data for it. Tech companies, on the other hand, possess the technical capability to implement AI and the agility to develop products rapidly.
This “government x business” co-creation model can also be applied to collaborations between private companies. For example, an SME lacking in-house expertise in building on-premises AI could partner with a local IT firm. The business provides operational knowledge and data, while the IT firm handles technical implementation. This opens a path to obtaining secure AI solutions without the need for large initial investments or sourcing highly skilled talent independently.
Practical Steps: How to Consider “On-Premises AI” for Your Company
Here are four concrete steps for executives and CTOs seriously considering on-premises AI implementation for their business operations.
Step 1: Map Tasks by “Confidentiality” and “Routine Nature”
First, evaluate all business tasks on two axes: “level of confidentiality” and “level of routine nature.” Tasks that are both highly confidential (e.g., customer personal information, financial data, intellectual property) and highly routine (can be manualized, have fixed document formats) are the top priority candidates for on-premises AI. VIEW CARD’s “pleading draft creation” fits this quadrant. On the other hand, tasks with low confidentiality but high routine nature (e.g., creating internal manuals, drafting internal reports) make cloud-based AI the first choice.
Step 2: Start with a Small-Scale PoC (Proof of Concept)
Making a large investment immediately is risky. First, select the smallest, most easily measurable task from the candidate list. For example, “automatic generation of notification letters based on specific contract clauses.” For this task, try running an open-source, lightweight AI model on a high-performance company PC or a test server. Using cloud services (like AWS PrivateLink or Azure Private Endpoint) to build a virtual private environment for the PoC is also a cost-effective option. At this stage, measure accuracy, speed, and, most importantly, “internal readiness for adoption.”
Step 3: Determine the Balance Between In-House Development and Outsourcing
If the PoC is successful, decide on the method for full-scale implementation. Evaluate if your company has the following resources:
- Infrastructure Expert: Personnel capable of server management and network configuration.
- AI Implementation Expert: Personnel capable of downloading and fine-tuning open-source AI models.
- Business Domain Expert: Personnel with a deep understanding of the tasks to be automated.
If these are all available, in-house development is possible. If not, outsourcing to a systems integrator or seeking a regional collaboration model like the aforementioned “government x business” type is realistic. Misjudging this can lead to “AI shelfware”—a system that is built but not used because it doesn’t fit the workflow.
Step 4: Establish Security and Governance Frameworks First
Running parallel to technical considerations, the most crucial step is designing governance. Even with on-premises systems, ultimate responsibility for AI-generated documents lies with humans. At a minimum, establish the following rules:
- Human Review Workflow: The responsible person and process for final verification and approval of AI-generated content.
- Usage Log Management System: Complete records of who used it, when, with what input, and what output was obtained.
- Model Update Policy: The frequency and procedure for retraining AI models to comply with new laws or internal regulations.
If technology advances ahead of governance, it can lead to significant risk exposure.
Cloud or On-Premises? The Answer Lies in Your Business Tasks
The VIEW CARD case shows that generative AI adoption points to a diverse future, not a one-size-fits-all “cloud-only” approach. It represents a move away from total dependence on SaaS and outsourcing. It reflects a management philosophy of evolving core data and business processes under one’s own control.
However, on-premises is not the answer for everything. For tasks requiring speed and flexibility, the advantages of cloud-based AI are overwhelming. The key is to deeply analyze your company’s operations and identify “where there is room for secure automation.”
AI is no longer an experimental technology. Now that financial institutions have begun practical implementation in core operations like debt management, it is time for executives in all industries to turn their attention to their own “highly confidential, routine tasks.” The first step begins not with the technical debate of cloud vs. on-premises, but with the managerial question: “Which tasks are our company’s treasure?”
Comments