TQ-18: Visa Secure Document Storage & Delivery¶

Overview¶

This document describes the implementation of encrypted S3 document storage for the Visa Management module, replacing the previous local filesystem storage, and the OTP-verified secure delivery workflow for issuing visas to customers.

Architecture¶

Two Workflows, One Encrypted Bucket¶

AGENT WORKFLOW                           CUSTOMER WORKFLOW
(Keycloak-authenticated)                 (Public page, no auth)

visamgmt.js                              visa-download.html
    |                                         |
    v                                         v
POST /visa/document/upload          POST /visa/delivery/verify
POST /visa/document/download             (email + phone match?)
    |                                         |
    v                                         v
VisaDocumentStorageService          POST /visa/delivery/sendotp
    |                                    (WhatsApp/SMS/email)
    v                                         |
S3 (tq-visa-documents)                       v
SSE-KMS encrypted                   POST /visa/delivery/validateotp
                                         (OTP correct?)
                                              |
                                              v
                                    Presigned URL (15 min)
                                    S3 GetObject → download

S3 Key Structure¶

tq-visa-documents/
  applications/
    {applicationId}/
      documents/
        {8-hex-suffix}.pdf      ← application documents (passport copies, photos)
        {8-hex-suffix}.jpg
        {8-hex-suffix}.pdf      ← issued visa PDF (via delivery/initiate)

No personal information in paths — only numeric application IDs and random suffixes.

Backend Components¶

Component	Location	Purpose
`SecureStorageConfig.java`	`tqapp/.../framework/media/`	JAXB config: bucket, prefix, KMS key, OTP settings
`VisaDocumentStorageService.java`	`tqapp/.../service/media/`	S3 upload/download/presign with SSE-KMS
`VisaDeliveryService.java`	`tqapp/.../entity/visa/`	Delivery orchestration: OTP generation, notifications, identity verification
`VisaAppFacade.java`	`tqapp/.../entity/visa/`	Entity CRUD + `findDeliveryByApplicationId`, `findApplicationByGuid`
`VisaApi.java`	`tqapi/.../api/`	7 new endpoints (2 document, 2 delivery agent, 3 delivery public)

API Endpoints¶

Endpoint	Access	Purpose
`POST /visa/document/upload`	agent, admin	Upload document to S3 (Base64 JSON)
`POST /visa/document/download`	agent, admin	Download document (binary response)
`POST /visa/delivery/initiate`	agent, admin	Upload issued visa PDF, create delivery record
`POST /visa/delivery/notify`	agent, admin	Send download link via email/SMS/WhatsApp
`POST /visa/delivery/verify`	guest	Customer identity verification (email + phone)
`POST /visa/delivery/sendotp`	guest	Send OTP to customer via chosen channel
`POST /visa/delivery/validateotp`	guest	Validate OTP, return presigned download URL

Frontend Components¶

Component	Purpose
`visamgmt.js`	Agent document upload (replaced `upload3()` with S3), download buttons
`visamgmt.html`	Download click handler with `getAuthHeaders()`, import for auth
`visa-download.html`	Public 4-step download page (outside Keycloak)
`visa-download.js`	Public page JS: verify → channel → OTP → download

Configuration¶

`config/nts-client.xml`¶

<SecureStorageConfig
    bucket="tq-visa-documents"
    prefix="applications/"
    kmsKeyId="arn:aws:kms:ap-south-1:ACCOUNT:key/KEY-ID"
    presignExpiryMinutes="15"
    maxFileSizeMb="10"
    allowedMimeTypes="application/pdf,image/jpeg,image/png"
    otpExpiryMinutes="10"
    otpLength="6"/>

Attribute	Description
`bucket`	S3 bucket name (must be created separately)
`prefix`	Base prefix for all visa documents
`kmsKeyId`	ARN of the KMS key used for SSE-KMS encryption
`presignExpiryMinutes`	Lifetime of presigned download URLs (default: 15)
`maxFileSizeMb`	Maximum upload file size (default: 10)
`allowedMimeTypes`	Comma-separated list of allowed MIME types
`otpExpiryMinutes`	OTP validity period (default: 10)
`otpLength`	Number of digits in OTP code (default: 6)

`config/api-roles.properties`¶

# Agent document endpoints
visa/document/upload=agent,admin
visa/document/download=agent,admin

# Agent delivery endpoints
visa/delivery/initiate=agent,admin
visa/delivery/notify=agent,admin

# Public customer endpoints
visa/delivery/verify=guest,agent,admin
visa/delivery/sendotp=guest,agent,admin
visa/delivery/validateotp=guest,agent,admin

Deployment Guide¶

1. AWS S3 Bucket Setup¶

Create the tq-visa-documents bucket in ap-south-1:

Block all public access (all 4 settings ON):

Block public access to buckets and objects granted through new ACLs          ON
Block public access to buckets and objects granted through any ACLs          ON
Block public access to buckets and objects granted through new policies       ON
Block public and cross-account access via any public bucket policies          ON

Default encryption: SSE-KMS with the dedicated key (see step 2).

Versioning: Enabled (allows recovery of accidentally overwritten documents).

Lifecycle rules: - Transition to S3 Infrequent Access after 90 days - Transition to S3 Glacier after 1 year

S3 access logging: Enabled (for audit trail).

CORS: Not configured (documents accessed via presigned URLs, not browser-direct).

2. AWS KMS Key Setup¶

Create a dedicated KMS key for visa document encryption:

Go to KMS -> Customer managed keys -> Create key
Key type: Symmetric
Key usage: Encrypt and decrypt
Alias: alias/tqpro-visa-documents
Key administrators: Your AWS admin users
Key users: The application IAM role/user (see step 3)

Note the key ARN (e.g. arn:aws:kms:ap-south-1:123456789012:key/abcd-1234-efgh-5678) and update the kmsKeyId attribute in nts-client.xml.

3. IAM Policy¶

Create policy TQProVisaDocS3Access and attach to the application's IAM role (EC2) or IAM user (local dev / CI):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisaDocBucketAccess",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::tq-visa-documents/*"
        },
        {
            "Sid": "VisaDocKMSAccess",
            "Effect": "Allow",
            "Action": [
                "kms:GenerateDataKey",
                "kms:Decrypt"
            ],
            "Resource": "arn:aws:kms:ap-south-1:ACCOUNT:key/KEY-ID"
        }
    ]
}

Note: kms:GenerateDataKey is required for SSE-KMS PutObject. kms:Decrypt is required for GetObject and presigned URL generation.

4. AWS Credential Configuration¶

The application uses DefaultCredentialsProvider — no credentials in config files. Configure per environment:

Environment	How
EC2 (production)	Attach IAM role to the instance. If an existing role is used, add the `TQProVisaDocS3Access` policy to it.
Local dev	Run `aws configure` with the service account access key. Verify: `aws s3 ls s3://tq-visa-documents --region ap-south-1`
CI (TeamCity)	Set `env.AWS_ACCESS_KEY_ID` and `env.AWS_SECRET_ACCESS_KEY` as secret build parameters

5. Twilio Configuration for OTP¶

OTP delivery uses the existing Twilio infrastructure configured in config/tourlinq.properties:

# Twilio credentials (already configured)
twilio.sid=ACe2a11f91a68f455eec0d61d8ed1005a2
twilio.token=<secret>
twilio.sms.sender=MGef576188087743a7fc0a783794961b45

SMS OTP: Uses MessageUtil.sendSMS() with the standard sender. No additional Twilio configuration required — OTP codes are sent as plain text messages.

WhatsApp OTP: Currently uses MessageUtil.sendSMS() as a fallback (sends SMS). For true WhatsApp delivery, a Twilio Content Template must be registered:

Go to Twilio Console -> Messaging -> Content Template Builder
Create a new template:
Name: visa_otp_code
Body: Your visa download verification code is: {{1}}. Valid for {{2}} minutes. Do not share this code.
Variables: {{1}} = OTP code, {{2}} = expiry minutes
Submit for WhatsApp approval (typically 24-48 hours)
Once approved, note the Content SID (e.g. HXabcdef1234567890)
Update VisaDeliveryService.sendOtp() to use MessageUtil.sendWhatsappMessage(phone, contentSid, params) instead of sendSMS()

WhatsApp Notification (delivery link): Similarly, register a template:

Name: visa_ready_download
Body: Dear {{1}}, your visa is ready for download. Please visit {{2}} to download your document. You will need to verify your identity.
Variables: {{1}} = customer name, {{2}} = download URL
Once approved, update VisaDeliveryService.notifyCustomer() to use the template SID

Until WhatsApp templates are registered and approved, the WhatsApp channel falls back to SMS delivery.

6. Email Template Configuration¶

Two email templates are stored in config/:

Template	Purpose	Variables
`visa-delivery-email.html`	Download link notification	`%CUSTOMER_NAME%`, `%DOWNLOAD_URL%`, `%LINKHOST%`
`visa-otp-email.html`	OTP verification code	`%OTP_CODE%`, `%LINKHOST%`

The %LINKHOST% variable is automatically replaced by MailUtil from the mail.targethost property in tourlinq.properties.

7. Database Migration¶

Run the migration script on each environment:

psql -h <host> -U <user> -d <database> -f config/db-changes/0037-visa-delivery-otp.sql

This adds 4 columns to nts.visadelivery: otp_attempts, otp_locked_until, verify_attempts, verify_locked_until. The script is idempotent (IF NOT EXISTS).

8. Public Page Access¶

The visa-download.html page must be accessible without Keycloak authentication. If using oauth2-proxy or a reverse proxy, exclude this path:

# nginx example: skip auth for visa download page
location = /visa-download.html {
    proxy_pass http://upstream;
    # No auth_request here
}
location = /js/modules/visa-download.js {
    proxy_pass http://upstream;
}

The page calls only guest-accessible API endpoints (visa/delivery/verify, sendotp, validateotp).

Security Design¶

Concern	Mitigation
PII in S3 paths	Only numeric `applicationId` and random hex suffix
Agent document access	Direct authenticated download — no URLs exposed in browser
Customer visa access	3-step verification: identity match -> OTP -> presigned URL
OTP brute force	Max 3 validation attempts per OTP; new OTP required after lockout
Identity verification brute force	Max 5 attempts, then 15-minute lockout
OTP storage	SHA-256 hashed in DB (truncated to 20 chars); never stored as plaintext
OTP timing attacks	Constant-time hash comparison
MIME type spoofing	Tika magic-byte detection; only PDF/JPEG/PNG allowed
File size abuse	10 MB limit enforced before S3 upload
Encryption at rest	SSE-KMS with dedicated key on all objects
Encryption in transit	HTTPS enforced by AWS SDK and webserver
Bucket exposure	All public access blocked at bucket level
Presigned URL sharing	15-minute expiry; download date recorded
Credential leak	DefaultCredentialsProvider — no secrets in config files
Phone number matching	Normalized (digits only), suffix match to handle country code variations

Migration from Local Filesystem¶

Existing visa documents stored on the local filesystem (legacy doc_link values like {guid}/{randomId}.pdf) continue to work during the transition:

visa/document/download checks if docUrl starts with applications/ (S3 key) or not (legacy path). Legacy files are read from the local filesystem.
A migration endpoint POST /visa/document/migrate (admin only) can be created to batch-migrate existing files to S3.

After migration is complete and verified, the backward-compatibility fallback can be removed.