From face detection and recognition models to a reusable, secure, cross-platform AI component.
Computer Vision · On-Device AI · Software Engineering
1. Introduction
Building a face recognition prototype has become relatively straightforward. With modern deep learning frameworks and pretrained models, developers can detect faces, extract embeddings, and compare identities using only a few lines of code.
However, turning those capabilities into a production-ready Software Development Kit (SDK) is a different engineering challenge.
A real-world SDK must do more than produce accurate predictions. It needs to operate consistently across applications, manage computational resources, expose stable interfaces, protect sensitive information, and support long-term maintenance.
At AIPractix, we have been developing AttFace, a face recognition engine designed for integration into real-world applications, including mobile attendance and identity verification systems.
In this article, we share the architecture, design decisions, and engineering considerations involved in building a reusable face recognition SDK.
Our focus is not on training a particular neural network. Instead, we explore what happens after the models are ready and how multiple AI components can be transformed into a practical software product.
2. A Face Recognition Model Is Not a Face Recognition SDK
A typical face recognition demonstration follows a simple pipeline:
- Detect a face in an image.
- Align or crop the detected region.
- Generate a face embedding.
- Compare the embedding against a reference.
This pipeline demonstrates the fundamental recognition capability, but real applications introduce additional requirements.
For example, an attendance system may need to reject blurred images, detect presentation attacks, work without internet access, and handle multiple devices.
An SDK must also provide predictable behavior when no face is detected, multiple faces are present, image quality is insufficient, or a model cannot be initialized.
These concerns belong to the software engineering layer around the AI models.
A production-ready SDK typically requires five major capabilities:
- AI pipeline: Reliable orchestration of face detection, alignment, recognition, and additional analysis.
- Runtime: Efficient inference and resource management.
- Public API: A consistent interface independent of implementation details.
- Security: Protection of sensitive data and control over SDK distribution.
- Integration: Practical support for application platforms and development workflows.
The most important architectural decision is to separate these responsibilities rather than expose individual model implementations directly to application developers.
3. Designing a Modular Face Analysis Pipeline
AttFace is organized around a native face analysis engine.
The engine integrates several computer vision capabilities:
| Component | Responsibility |
|---|---|
| Face Detection | Locate faces and return bounding boxes |
| Face Landmarks | Estimate facial keypoints |
| Face Alignment | Normalize the face region for recognition |
| Face Recognition | Generate identity embeddings |
| Face Liveness | Estimate whether an input may represent a presentation attack |
| Face Quality | Evaluate input suitability |
| Head Pose | Estimate facial orientation |
These components are related, but they do not always need to run together.
For example, a photo management application may require detection and embedding extraction, while a biometric attendance application may additionally require liveness and quality checks.
Running every model for every request can create unnecessary computational overhead.
3.1 Configurable processing
A useful design is to support component-specific processing policies.
For example:
- Disabled: Skip the component.
- Evaluate: Run the component and return its result.
- Required: Run the component and enforce an application-defined acceptance condition.
The distinction between evaluation and enforcement is important.
An application may want to inspect head pose without rejecting a face, while an enrollment workflow may require a face to remain within a specific orientation range.
Similarly, liveness can be evaluated separately from the decision to accept or reject an authentication attempt.
This separation allows a single engine to support different workflows without requiring different AI implementations.
3.2 Structured analysis results
An SDK should return structured information rather than only a Boolean success or failure value.
A conceptual result might contain:
{
"faces": [
{
"boundingBox": [120, 80, 260, 260],
"recognition": {
"embeddingAvailable": true
},
"liveness": {
"score": 0.93,
"status": "evaluated"
},
"quality": {
"status": "acceptable"
},
"headPose": {
"yaw": 4.2,
"pitch": -2.1,
"roll": 1.3
}
}
]
}
Illustrative API structure and values; not a literal AttFace SDK response.
A structured result offers several advantages:
- Applications can implement their own business rules.
- Developers can distinguish model outputs from policy decisions.
- Integration tests become easier to maintain.
- Additional analysis components can be introduced through versioned interfaces.
The SDK should also provide explicit error codes for initialization failures, unsupported inputs, expired licenses, and inference errors.
These are software errors, not face analysis results, and should be handled separately.
4. Face Detection, Alignment, and Recognition
4.1 Face detection
Face detection is the entry point to the recognition pipeline.
AttFace uses an SCRFD-based detector as part of its face analysis architecture.
The detector identifies facial regions and provides the information needed for subsequent processing.
In practice, choosing a detector involves balancing several factors:
- Detection robustness under different lighting conditions.
- Small or partially visible faces.
- Non-frontal facial orientations.
- Computational cost.
- Compatibility with downstream alignment.
Detector performance directly affects the quality of subsequent recognition.
A recognition model cannot recover facial information that was incorrectly cropped or excluded during detection.
4.2 Facial landmarks and alignment
Face alignment is a critical but sometimes underestimated component.
Modern face recognition models generally benefit from consistent facial positioning and scale.
In the AttFace pipeline, facial landmarks can be used to estimate key facial reference points, including the eyes, nose, and mouth.
These reference points support geometric normalization before embedding extraction.
A typical alignment workflow consists of:
- Detecting the face.
- Estimating facial landmarks.
- Selecting reference landmarks.
- Computing a geometric transformation.
- Generating the aligned face image.
The selected alignment strategy should remain consistent between enrollment and recognition.
Even when the embedding model stays unchanged, changes in alignment behavior may alter similarity scores and require threshold revalidation.
This has an important consequence for SDK maintenance: the recognition pipeline should be versioned as a whole, not only by the embedding model name.
4.3 Face embeddings
Instead of storing an identity as a class label, modern face recognition systems commonly represent facial identity using a numerical embedding.
Let two face embeddings be:
A commonly used similarity function is cosine similarity:
The application can compare the resulting score against a calibrated threshold.
However, the similarity score is not a universal probability that two faces belong to the same person.
The appropriate threshold depends on the model, preprocessing pipeline, evaluation data, intended operating conditions, and acceptable false match and false non-match rates.
Therefore, embedding extraction and identity verification policy should remain conceptually separate.
For a reusable SDK, this makes it possible to support different applications without hard-coding a single universal acceptance threshold.
5. Integrating Face Liveness Detection
Face recognition answers the question:
Do these two facial representations appear to belong to the same identity?
It does not, by itself, determine whether a camera is observing a physically present person.
A system may encounter presentation attacks involving printed photographs, replayed videos, or other artifacts.
Face liveness detection, more formally discussed within presentation attack detection (PAD), attempts to identify suspicious presentation characteristics.
AttFace incorporates a liveness analysis component into its face processing pipeline.
However, liveness estimation should not be described as a guarantee against every attack.
Its effectiveness depends on the threat model, sensor characteristics, attack materials, and evaluation conditions.
5.1 Different workflows need different policies
Consider two use cases:
Employee enrollment
The application may require adequate image quality, acceptable facial orientation, and a liveness result before enrolling a face.
Routine face analysis
The application may only need detection and recognition, with liveness disabled to reduce processing requirements.
A configurable pipeline makes both scenarios possible without duplicating the recognition engine.
5.2 Avoid treating liveness as a single magic threshold
In production, developers should consider:
- False acceptance of attack presentations.
- False rejection of genuine users.
- Device and camera variation.
- Lighting and environmental conditions.
- Interaction between quality checks and PAD performance.
Thresholds must be selected using relevant evaluation data, not arbitrary values.
Liveness should also be combined with appropriate application security controls rather than treated as the sole defense.
6. Building a Portable Native Core
One of the main architectural goals of AttFace is to reuse the same core technology across platforms.
Instead of independently implementing recognition pipelines in Android, iOS, and other environments, the engine can be organized around a native C++ core.
A simplified architecture looks like this:
The native core owns the main AI processing logic, while platform-specific wrappers handle application integration.
This separation provides several benefits.
6.1 Consistent behavior across platforms
A shared implementation reduces the risk of algorithmic differences between Android and iOS.
Model preprocessing, alignment, threshold interpretation, and result generation can follow the same core logic.
Platform-specific differences still need to be tested, particularly around image formats, orientation, memory handling, and runtime behavior.
6.2 Easier model updates
If the inference implementation changes, the update can be made inside the core without redesigning the entire application-facing API.
For commercial SDKs, this is important because customer applications may follow different release schedules.
6.3 Controlled implementation exposure
A compiled native library can provide a stable integration surface without requiring the application developer to receive the complete source code of the underlying engine.
This is not equivalent to perfect intellectual property protection, but it supports a practical commercial distribution model.
7. Android and iOS SDK Integration
The value of a reusable native core becomes clearer when integrating it into mobile applications.
7.1 Android integration
On Android, a native C++ implementation can be exposed through the Java Native Interface (JNI).
The integration stack typically looks like:
The SDK can be packaged as an Android Archive (AAR), allowing application developers to consume it as a dependency.
The JNI layer should remain thin.
Its primary responsibilities include:
- Converting image input into the format expected by the core.
- Managing native object lifetimes.
- Translating configuration and result structures.
- Propagating errors to the application layer.
Complex AI logic should remain inside the native engine rather than being duplicated in JNI.
7.2 iOS integration
On iOS, the native implementation can be distributed using a framework package such as an XCFramework.
A platform bridge exposes the native capabilities to Swift or Objective-C applications.
Important integration considerations include:
- Binary architecture compatibility.
- Image buffer conversion.
- Memory ownership.
- Thread safety.
- Framework distribution and versioning.
The objective is to make the SDK feel natural to the host platform while preserving the shared AI behavior.
7.3 Flutter and additional platforms
A reusable native core also makes it practical to support cross-platform frameworks.
For example, a Flutter plugin can expose a Dart API and delegate native processing to the Android and iOS SDKs.
The same principles may later be extended to other platforms where an appropriate native integration layer is available.
The important point is that cross-platform support should be an integration concern, not a reason to duplicate AI implementations.
8. Performance Engineering for On-Device AI
A face recognition SDK may execute several neural networks during a single analysis request.
Total processing time can be approximated as:
Not all components must run in every request, and the actual pipeline may reuse intermediate results.
On mobile devices, latency is only one part of performance.
Developers must also consider:
- Peak memory consumption.
- Model initialization time.
- CPU utilization.
- Thermal behavior.
- Battery impact.
- Camera-to-inference data transfer.
- Repeated analysis under sustained workloads.
8.1 Load models deliberately
Repeatedly constructing inference sessions for every frame can create unnecessary overhead.
A production design should establish model lifetimes explicitly and reuse initialized resources where appropriate.
8.2 Avoid unnecessary processing
Running a recognition network when no usable face is present wastes computation.
Similarly, an application may not need full landmark estimation, liveness evaluation, and embedding extraction for every preview frame.
A configurable pipeline allows processing costs to follow the application's requirements.
8.3 Keep expensive processing off the UI thread
Mobile SDK operations should support application architectures that keep expensive inference tasks away from the main UI thread.
For continuous camera input, applications may need frame sampling, backpressure, and explicit control over concurrent analysis requests.
8.4 Benchmark end-to-end behavior
A useful SDK benchmark should distinguish model inference time from end-to-end processing time.
For example:
| Metric | Why it matters |
|---|---|
| Initialization latency | Application startup experience |
| Detection latency | Camera responsiveness |
| Full analysis latency | User interaction time |
| Peak memory | Device compatibility |
| Sustained processing | Thermal and battery behavior |
| Failure rate | Operational reliability |
Performance should be reported with device specifications, input resolution, runtime configuration, workload, and measurement methodology.
A single inference-time number without this context can be misleading.
9. SDK Security, Privacy, and Licensing
Face recognition systems handle sensitive biometric information, so privacy and security must be considered from the beginning.
9.1 On-device processing
An on-device SDK can process facial images and generate embeddings locally without requiring a cloud inference request for each analysis.
This can offer meaningful benefits:
- Lower dependency on network availability.
- Reduced transmission of raw facial images.
- More control over application data flows.
- Lower network-related latency.
However, local processing alone does not guarantee privacy.
The host application remains responsible for defining how biometric information is stored, retained, synchronized, protected, and deleted.
Face embeddings should also be treated as sensitive biometric data rather than harmless anonymous identifiers.
9.2 SDK licensing
Commercial SDK distribution introduces another engineering requirement: controlling authorized usage.
AttFace's licensing architecture uses cryptographic signature verification, including Ed25519-based verification.
In a signed-license design, a licensing service can issue license data signed with a private key, while the distributed SDK verifies the signature using an embedded public key.
The private signing key must remain under server-side control.
This approach separates license issuance from license verification and can support designs where verification does not require constant network access.
Nevertheless, offline licensing introduces important challenges.
A production licensing design must consider:
- License scope and expiration.
- Device or application binding.
- Replay and rollback resistance.
- Updates and revocation.
- Clock manipulation.
- Recovery after device replacement.
- Protection against application tampering.
No client-side licensing mechanism can guarantee complete protection against a fully compromised device.
The goal is to establish practical controls without making legitimate customer integration unnecessarily difficult.
9.3 Data protection
SDK and application developers should define explicit data ownership and responsibility boundaries.
Important measures include encryption where appropriate, secure key storage, access controls, retention limits, and mechanisms to support deletion requests.
The SDK should avoid unnecessary collection or transmission of identifiable information.
Security is not just an additional feature. It affects the design of the public API, the storage model, the licensing architecture, and the deployment workflow.
10. Designing an SDK That Developers Can Actually Use
A powerful AI engine can still be difficult to adopt if integration requires deep knowledge of its internals.
A useful SDK should provide a compact, predictable interface.
Conceptually, developers should be able to:
- Initialize the engine.
- Configure enabled analysis components.
- Provide an image.
- Receive structured analysis results.
- Compare compatible face embeddings.
- Handle errors and release resources.
For example, a platform-independent API may expose operations conceptually equivalent to:
initialize(configuration)
analyze(image)
compare(embeddingA, embeddingB)
updateLicense(license)
release()
Conceptual API only; language-specific SDK interfaces may differ.
Several additional design choices are important.
Stable data contracts: Public result structures should be versioned carefully to avoid breaking customer applications.
Clear error handling: Integration failures should produce actionable error information.
Lifecycle management: Initialization and resource cleanup must have well-defined semantics.
Thread safety: The SDK should document whether an engine instance can be shared across threads.
Compatibility management: Embedding model and preprocessing changes must not silently invalidate existing face databases.
Documentation: Developers need working examples, platform requirements, release notes, and troubleshooting guidance.
The actual quality of an SDK is often revealed not during the first successful integration, but during upgrades, production debugging, and long-term maintenance.
11. Testing Beyond Model Accuracy
Model evaluation and SDK validation are related but different activities.
A recognition model may perform well in offline experiments while its application integration fails because of image orientation, memory handling, incompatible binaries, or unsupported device configurations.
Production readiness requires several levels of validation.
Algorithm validation
- Detection and landmark behavior.
- Recognition accuracy under relevant conditions.
- Threshold calibration.
- Liveness evaluation against defined attack types.
- Quality and head pose behavior.
Integration validation
- Android and iOS compatibility.
- Correct image orientation and pixel formats.
- Native memory ownership.
- Repeated initialization and cleanup.
- Error handling.
- Concurrent execution where supported.
Operational validation
- Long-running camera sessions.
- Resource consumption.
- Application lifecycle changes.
- SDK updates.
- License renewal and failure scenarios.
- Regression testing across model versions.
An effective release process should track SDK binaries, model versions, supported environments, and regression results.
The goal is not merely to verify that the models work, but to ensure the entire software component behaves predictably under real application conditions.
12. Lessons From Building AttFace
Developing AttFace has reinforced several engineering principles.
Lesson 1: The pipeline is the product
Face detection and recognition models are essential components, but a commercial SDK is defined by the entire processing pipeline, its interfaces, and its behavior under real-world constraints.
Lesson 2: Modularity creates flexibility
Different applications require different combinations of recognition, liveness, quality, and head pose analysis.
A configurable architecture is more reusable than a single rigid pipeline.
Lesson 3: Stable interfaces matter as much as model performance
Model improvements are valuable, but customers also depend on compatibility, predictable outputs, and reliable upgrades.
Lesson 4: Deployment constraints should influence architecture early
On-device processing introduces challenges around memory, performance, threading, binary distribution, and platform compatibility.
These concerns are easier to address during initial design than after multiple applications have integrated the SDK.
Lesson 5: Production readiness is an ongoing process
A production-oriented architecture is not a one-time achievement.
It requires continuous evaluation, testing, security review, documentation, and compatibility management as devices, models, and customer requirements evolve.
13. When Should You Choose an SDK Instead of a Server-Based System?
An on-device SDK is particularly useful when applications benefit from local inference, offline operation, and direct integration into their existing software.
However, not every face recognition application is best served by an SDK-only design.
A centralized server-based architecture may be more appropriate when the system requires:
- A large shared face database.
- Centralized identity management.
- Matching across many applications or devices.
- Server-controlled inference resources.
- Centralized monitoring and operational management.
A hybrid design is also possible, with some processing performed locally and other operations handled by an internal server.
The right architecture depends on the application's scale, privacy requirements, latency targets, operating environment, and maintenance constraints.
In the next article, Building a Production-Ready Face Recognition System Using an Internal Server, we will explore the server-side alternative, including inference services, face database management, API design, deployment, and scalability.
14. Conclusion
Building a production-ready face recognition SDK requires much more than integrating a collection of neural networks.
The real engineering work lies in creating a modular processing pipeline, designing stable APIs, optimizing execution across devices, managing sensitive information, and maintaining compatibility over time.
Through the development of AttFace, AIPractix continues to explore how face recognition and other computer vision technologies can be packaged into practical components for real-world software systems.
Our broader goal is to make AI integration more accessible to companies that need specialized capabilities without building every component from scratch.
