W-14: Data Team Ways of Work¶
| Field | Value |
|---|---|
| Document | W-14 |
| Title | Data Team Ways of Work |
| Status | Draft |
| Owner | Data Lead (TBD) |
| Created | 2026-04-05 |
| Review | Quarterly |
| Depends On | W-01 (Company Operating Rhythm), STD-DATA-056 (Data Quality Monitoring), STD-DATA-057 (Master Data Management), STD-DATA-058 (Data Catalogue), STD-DATA-064 (Data Retention), STD-GOV-131 (Data Classification), DA-06 (Cross-Border Data Architecture), PII Handling Standard, Database Schema Change Management Standard |
Purpose¶
Define how Simpaisa's Data team operates: how we govern data, maintain quality, manage retention, serve analytics requests, and enforce data policies across all seven markets. This is the single source of truth for data team process.
This document applies to the Data team within the CDO division and to all Simpaisa teams that create, process, or consume data — which is every team.
Team Structure¶
Data Lead — TBD (reports to CDO)
├── Data Engineer (1-2)
│ └── Pipeline development, ETL, data warehouse maintenance
│
├── Data Analyst (1-2)
│ └── Dashboards, reports, analytics requests, data quality monitoring
│
└── Data Governance Analyst (1)
└── Classification enforcement, retention compliance, catalogue maintenance
Note: The Data Lead role is currently being recruited. Until filled, the CDO acts as interim Data Lead. Data governance responsibilities are shared across the team with CDO oversight.
Cross-functional dependencies: - Engineering teams: Own the source databases and services that produce data. - InfoSec (Hamza Bari): Collaborates on data classification and PII handling. - Operations: Primary consumers of transaction dashboards and settlement reports. - Finance: Consumers of revenue, settlement, and reconciliation data. - Commercial: Consumers of merchant analytics and pipeline data.
1. Data Governance Meeting Cadence¶
| Meeting | Frequency | Duration | Attendees | Purpose |
|---|---|---|---|---|
| Data Governance Forum | Monthly (3rd Tuesday) | 60 min | Data Lead (chair), InfoSec representative, Engineering team lead (rotating), Operations representative, Finance representative | Data quality review, policy compliance, classification issues, cross-team data matters |
| Data Quality Standup | Weekly (Thursday) | 15 min | Data team | Data quality alerts, pipeline health, open requests |
| Data & CDO Sync | Fortnightly | 30 min | Data Lead + CDO | Strategic priorities, resourcing, escalations, governance decisions |
Data Governance Forum Agenda¶
| Segment | Duration | Content |
|---|---|---|
| Data quality scorecard review | 15 min | Quality metrics by domain, SLA compliance, open issues |
| Retention compliance status | 10 min | Expiring data, deletion jobs status, audit readiness |
| Classification and PII matters | 10 min | New data sources requiring classification, PII handling issues |
| Cross-team data requests | 15 min | New analytics requests, data sharing proposals, access requests |
| Actions and decisions | 10 min | Decisions recorded, actions assigned |
Output: Meeting minutes stored in Governance/Data-Governance-Minutes/. Decisions that affect architecture escalated to ARB.
2. Data Quality Monitoring¶
Per STD-DATA-056, Simpaisa monitors data quality across all critical data domains.
Data Quality Dimensions¶
| Dimension | Definition | How Measured | Target |
|---|---|---|---|
| Completeness | Required fields populated | Automated checks on key tables | ≥ 99.5% for transaction data |
| Accuracy | Data matches real-world values | Reconciliation against source systems | ≥ 99.9% for financial data |
| Timeliness | Data available within expected window | Pipeline latency monitoring | Transaction data available within 5 minutes |
| Consistency | Same data matches across systems | Cross-system reconciliation | Zero unresolved discrepancies for settlement data |
| Uniqueness | No unintended duplicates | Duplicate detection rules | Zero duplicate transactions in production |
| Validity | Data conforms to defined formats and ranges | Schema validation, range checks | ≥ 99.9% |
Critical Data Domains¶
| Domain | Owner (Engineering) | Data Lead Oversight | Quality Checks |
|---|---|---|---|
| Transactions (pay-in, pay-out) | Pay-In / Pay-Out teams | Data team monitors | Completeness, accuracy, uniqueness, timeliness |
| Settlements | Operations + Finance | Data team monitors | Accuracy, consistency (reconciliation vs partner) |
| Merchant master data | Portal team | Data Governance Analyst | Completeness, validity, uniqueness |
| Customer / payer data | Pay-In team | Data Governance Analyst + InfoSec | Completeness, validity, PII classification |
| Partner / channel configuration | DevOps + Operations | Data team monitors | Validity, consistency |
| User access and audit logs | Engineering (all teams) | InfoSec + Data team | Completeness, timeliness, immutability |
Alerting and Escalation¶
| Alert Level | Condition | Action | Escalation |
|---|---|---|---|
| Warning | Quality metric drops below target but above critical threshold | Data team investigates. Logged in quality tracker. | Data Lead |
| Critical | Quality metric drops below critical threshold or reconciliation discrepancy detected | Immediate investigation. Source team notified. | Data Lead → CDO if unresolved within 4 hours |
| Incident | Data corruption, loss, or incorrect financial data in production | Incident response triggered (per W-12 incident process) | Data Lead → CDO → CISO (if breach suspected) |
Quality Reporting¶
- Weekly: Data quality scorecard produced by Data Analyst. Shared at Data Quality Standup.
- Monthly: Quality trends presented at Data Governance Forum.
- Quarterly: Data quality summary included in CDO's report to the Technology & InfoSec Board Committee.
3. Data Retention Enforcement¶
Per STD-DATA-064, all Simpaisa data has a defined retention period based on regulatory requirements across all seven markets.
Retention Schedule (Summary)¶
| Data Category | Minimum Retention | Maximum Retention | Governing Regulation | Deletion Method |
|---|---|---|---|---|
| Transaction records | 10 years | 10 years | SBP (PK), BB (BD), NRB (NP), CBI (IQ), CBUAE (AE), SAMA (KSA) — longest applies | Automated deletion job |
| KYC / identity data | 10 years after relationship ends | 10 years after relationship ends | AML regulations (all markets) | Automated deletion job + audit log |
| Audit logs | 7 years | 7 years | Internal policy (aligned to most stringent market) | Automated archival then deletion |
| Application logs (non-audit) | 90 days | 1 year | Internal policy | Automated rotation and deletion |
| Analytics / aggregated data | Indefinite (anonymised) | Indefinite | N/A (no PII) | N/A |
| Session data, temporary caches | 24 hours | 7 days | Internal policy | Automatic TTL expiry |
Enforcement Process¶
- Automated deletion jobs run monthly for each data category.
- Pre-deletion verification: Before deletion, the job logs what will be deleted (record counts, date ranges). Data Governance Analyst reviews the log.
- Deletion execution: Job runs in a maintenance window (Saturday 02:00 UTC).
- Post-deletion verification: Record counts verified. Deletion logged in the retention audit trail.
- Exception handling: If a regulatory hold or legal hold prevents deletion, the Data Governance Analyst marks the affected records as "held" with a reason and expiry date.
Compliance Verification¶
- Monthly: Data Governance Analyst verifies deletion jobs ran successfully. Results reported at Data Governance Forum.
- Quarterly: Retention compliance summary included in the CDO's board report.
- Annually: External audit reviews retention compliance as part of the annual audit cycle.
4. Master Data Management¶
Per STD-DATA-057, Simpaisa maintains master data for key entities to ensure a single source of truth.
Master Data Entities¶
| Entity | System of Record | Owner | Governance |
|---|---|---|---|
| Merchants | Merchant Portal database | Portal team | Data Governance Analyst verifies completeness and uniqueness monthly |
| Partners / Channels | Channel configuration service | DevOps + Operations | Operations validates. Data team monitors for consistency. |
| Countries / Markets | Reference data service | Data team | Static data. Reviewed annually or when new market added. |
| Currency / FX rates | Treasury / Finance system | Finance | Data team consumes. Finance owns accuracy. |
| Product / Service catalogue | Product configuration | CPO / Product team | Data team ensures catalogue aligns with actual deployed services. |
| Regulatory entities (regulators, licence types) | Architecture repo (reference data) | CDO / Regulatory | Updated when regulatory landscape changes. |
Rules¶
- Each master data entity has exactly one system of record. All other systems consume from it.
- Changes to master data follow a controlled process: request → review → approve → apply.
- Bulk updates to master data require Data Lead approval.
- Master data quality is monitored as part of the data quality framework (section 2).
5. Analytics Request Process¶
All Simpaisa teams may request dashboards, reports, or ad-hoc analytics from the Data team.
Request Types¶
| Type | Definition | Turnaround SLA | Approval |
|---|---|---|---|
| Ad-hoc query | One-off data question (e.g., "How many transactions in Pakistan last month?") | 2 business days | None required |
| New dashboard | Recurring visual report in BI tool | 10 business days | Data Lead approves scope |
| Dashboard modification | Change to existing dashboard (new metric, filter, layout) | 5 business days | None required |
| New scheduled report | Automated report delivered by email or Slack | 10 business days | Data Lead approves scope |
| Data extract | One-off export of data for a specific purpose | 3 business days | Data Lead approves (PII extracts require InfoSec approval) |
| New data pipeline | New ETL job to bring data from source to warehouse | Per weekly planning | Data Lead + CDO approve |
How to Submit a Request¶
- Create a ticket in the project's issue tracker with label
data-request. - Include: what data is needed, why, who will use it, desired format, frequency (one-off or recurring), and any deadline.
- Data team triages at the weekly Data Quality Standup (Thursday).
- Requestor receives acknowledgement within 1 business day and estimated delivery date within 2 business days.
Prioritisation¶
Data requests are prioritised by the Data Lead based on: 1. Regulatory or compliance requirement — highest priority. 2. Revenue impact — requests tied to revenue or settlement accuracy. 3. Operational efficiency — requests that reduce manual work. 4. Strategic insight — requests tied to CDO or ELT strategic questions. 5. Nice to have — lowest priority. Addressed when capacity allows.
Rules¶
- No direct database access for non-engineering staff. All data access through dashboards, reports, or approved extracts.
- All data extracts containing PII must be approved by InfoSec and logged.
- Dashboard access follows role-based access control. Data Lead defines who sees what.
6. Data Catalogue Maintenance¶
Per STD-DATA-058, Simpaisa maintains a data catalogue that documents all significant data assets.
What Is Catalogued¶
| Asset Type | Catalogued Information |
|---|---|
| Databases (production) | Name, purpose, technology, owner, data classification, retention policy, backup schedule |
| Key tables / collections | Name, description, schema summary, PII fields flagged, row count estimate, update frequency |
| Data pipelines (ETL) | Name, source, destination, schedule, owner, SLA |
| Dashboards and reports | Name, purpose, data sources, owner, audience, refresh frequency |
| External data feeds | Partner name, data received, format, frequency, SLA, contract reference |
| APIs that expose data | Endpoint, data returned, classification of data, authentication required |
Maintenance Process¶
| Task | Frequency | Owner |
|---|---|---|
| Update catalogue for new databases, tables, or pipelines | Within 5 business days of deployment | Engineering team (source) + Data Governance Analyst (catalogue entry) |
| Review catalogue completeness | Quarterly | Data Governance Analyst |
| Verify PII field flags against actual data | Quarterly | Data Governance Analyst + InfoSec |
| Archive entries for decommissioned assets | On decommission | Data Governance Analyst |
Rules¶
- Every new database or significant table must have a catalogue entry before production deployment. This is a gate in the deployment checklist.
- PII fields must be explicitly flagged in the catalogue. Missing PII flags are treated as a data governance incident.
- The catalogue is the authoritative source for answering "what data do we have and where is it?"
7. Data Classification Enforcement¶
Per STD-GOV-131, all Simpaisa data is classified into one of four levels.
Classification Levels¶
| Level | Label | Definition | Examples | Handling |
|---|---|---|---|---|
| 1 | Public | Information intended for public consumption | Marketing materials, public API documentation | No restrictions |
| 2 | Internal | Business information not intended for public release | Internal reports, staff directories, non-sensitive configs | Access limited to Simpaisa staff |
| 3 | Confidential | Sensitive business or personal data | Merchant contracts, transaction volumes, employee PII, partner agreements | Encrypted at rest and in transit. Access logged. Need-to-know basis. |
| 4 | Restricted | Highly sensitive data with severe impact if disclosed | Card numbers (PAN), bank account details, KYC documents, authentication credentials, encryption keys | Encrypted at rest and in transit. Access logged and alerted. Minimal retention. Tokenised where possible. |
Enforcement¶
- At creation: Every new data field, table, or data store must be classified by the creating team before production deployment.
- In the catalogue: Classification level recorded for every catalogued asset (section 6).
- In code: PII and restricted fields annotated in schema definitions (e.g., comments, tags).
- In transit: Data classification determines minimum TLS version and whether additional encryption is required.
- In dashboards: Restricted data is never displayed in dashboards. Confidential data displayed only to authorised roles.
- In logs: Restricted and confidential data must never appear in application logs. Log scrubbing enforced.
Violations¶
Data classification violations (e.g., restricted data in logs, unclassified PII in a new table) are reported to the Data Lead and InfoSec. Severity depends on data level and exposure:
| Violation | Severity | Response |
|---|---|---|
| Restricted data in logs or unencrypted storage | Critical | Immediate remediation. Incident response if exposure confirmed. |
| Unclassified PII in production | High | Classify within 48 hours. Review access controls. |
| Confidential data accessible without proper authorisation | High | Access revoked immediately. Access review within 5 business days. |
| Missing classification on new data store | Medium | Classify within 5 business days of discovery. |
8. PII Handling Protocols¶
Per the PII Handling Standard, all teams that process personal data follow these protocols.
PII Definition at Simpaisa¶
PII includes: names, phone numbers, email addresses, national identity numbers (CNIC, NID, etc.), dates of birth, addresses, bank account numbers, card numbers, KYC documents (photos, scans), biometric data, device identifiers linked to individuals.
Handling Rules¶
| Rule | Requirement |
|---|---|
| Collection | Collect only the PII required for the specific business purpose. No speculative collection. |
| Storage | Encrypted at rest (AES-256 or equivalent). Stored in designated databases with access controls. |
| Transit | Encrypted in transit (TLS 1.2 minimum, TLS 1.3 preferred). |
| Access | Role-based access control. Access logged. Quarterly access review by Data Governance Analyst + InfoSec. |
| Display | Masked by default in UIs (e.g., show last 4 digits of phone number). Full display only when operationally necessary and access-controlled. |
| Logging | PII must never appear in application logs. Log scrubbing rules enforced in logging framework configuration. |
| Sharing | PII shared with third parties only under a data processing agreement. Approved by InfoSec and Legal. |
| Deletion | Per retention schedule (section 3). Right-to-deletion requests processed within 30 days where regulation permits. |
| Tokenisation | Card numbers (PAN) are tokenised. Full PAN never stored in Simpaisa systems (PCI DSS compliance). |
| Cross-border transfer | PII transferred across borders only per approved data flows (section 10). |
PII Incident Response¶
If PII is exposed, leaked, or mishandled: 1. Report immediately to Data Lead and InfoSec. 2. InfoSec assesses scope per W-12 (Security Operations) data breach response process. 3. Data Lead coordinates data-side remediation (access revocation, deletion of exposed copies). 4. Regulatory notification per W-12 section 7 (jurisdiction-specific timelines).
9. Database Schema Change Process¶
Per the Database Schema Change Management Standard, all changes to production database schemas follow a controlled process.
Change Categories¶
| Category | Definition | Examples | Approval |
|---|---|---|---|
| Non-breaking additive | New column (nullable), new index, new table | Adding an optional field to transaction record | Team lead approves |
| Breaking change | Column rename, column removal, type change, constraint change | Changing a field from nullable to required | Team lead + Data Lead approve |
| New database | Entirely new database instance | New analytics database, new service database | ADR + ARB approval (per W-13 section 8) |
| Migration | Moving data between databases or restructuring existing schema | Splitting a monolithic database into service-specific databases | ADR + ARB approval |
Process¶
1. Author Engineer writes migration script (versioned, in repo)
2. Review PR review by team lead + Data team (for breaking/new/migration)
3. Test Migration tested against staging environment
4. Catalogue Data catalogue updated with schema changes
5. Classify New fields classified per STD-GOV-131
6. Deploy Migration applied in production during approved window
7. Verify Post-deployment verification (row counts, data integrity)
Rules¶
- All schema changes are version-controlled migration scripts. No manual DDL in production.
- Breaking changes require a migration plan that addresses all consuming services and pipelines.
- New tables or columns containing PII must be flagged in the migration PR and reviewed by InfoSec.
- Schema changes to financial data (transactions, settlements) require Data Lead sign-off.
- Rollback plan required for all breaking changes before deployment approval.
10. Cross-Border Data Flow Reviews¶
Per Data Architecture DA-06, Simpaisa processes data across seven markets. Any change to cross-border data flows requires review.
Current Data Flow Summary¶
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Pakistan │ │Bangla- │ │ Nepal │
│ (PK) │────▶│ desh(BD)│ │ (NP) │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
▼ ▼ ▼
┌──────────────────────────────────────────┐
│ Central Processing (AWS) │
│ (Dubai / Middle East region) │
└──────────────────────────────────────────┘
▲ ▲
│ │
┌────┴────┐ ┌────┴────┐
│ Iraq │ │ UAE │
│ (IQ) │ │ (AE) │
└─────────┘ └─────────┘
│
┌─────┴─────┐
│ KSA │
│ (KSA) │
└───────────┘
Review Triggers¶
A cross-border data flow review is required when:
| Trigger | Example | Reviewer |
|---|---|---|
| New data flow between markets | Pakistan transaction data needed in UAE for reconciliation | Data Lead + InfoSec + Legal |
| New data category crossing borders | KYC images being centralised from local storage | Data Lead + InfoSec + Legal + CISO |
| Change in data residency | Moving a database from one AWS region to another | Data Lead + CISO + CDO + ARB |
| New partner requiring data transfer | New banking partner in KSA needing transaction data | Data Lead + InfoSec + Legal + Commercial |
| Regulatory change affecting data flows | New localisation requirement in a market | Data Lead + Legal + CDO |
Review Process¶
- Request: Team submits data flow change request to Data Lead with: source market, destination market, data categories, data classification levels, purpose, legal basis.
- Impact assessment: Data Lead and InfoSec assess regulatory requirements for both source and destination markets.
- Legal review: Legal confirms data transfer is permissible under applicable regulations.
- Controls verification: Confirm encryption, access controls, audit logging, and data processing agreements are in place.
- Approval: Data Lead + CDO approve (or escalate to ARB for significant changes).
- Documentation: Approved data flow documented in the cross-border data flow register.
- Monitoring: New data flow added to ongoing monitoring and included in quarterly compliance review.
Market-Specific Considerations¶
| Market | Key Data Residency Requirement | Regulator |
|---|---|---|
| Pakistan (PK) | Transaction data processed locally or under SBP-approved arrangement | State Bank of Pakistan (SBP) |
| Bangladesh (BD) | MFS data subject to Bangladesh Bank data localisation directives | Bangladesh Bank (BB) |
| Nepal (NP) | Payment data subject to NRB oversight | Nepal Rastra Bank (NRB) |
| Iraq (IQ) | Mobile money data subject to CBI regulations | Central Bank of Iraq (CBI) |
| UAE (AE) | Data protection per federal data protection law. CBUAE oversight for financial data. | Central Bank of UAE (CBUAE) |
| KSA (KSA) | PDPL applies. SAMA cybersecurity framework for financial data. | Saudi Central Bank (SAMA) |
Rule: When in doubt about whether a data flow is permissible, stop and ask. The Data Lead and Legal must confirm before any new cross-border data transfer is implemented.
Compliance with This Document¶
All Simpaisa teams that create, process, or consume data must follow these processes. The Data team is responsible for enforcement and monitoring. This document is reviewed quarterly by the Data Lead (or CDO as interim) and approved at the Data Governance Forum.
Changes to this document follow the standard Architecture Practice process (per W-13): propose via markdown PR, review at ARB or Data Governance Forum, merge on approval.