GDPR uncertainty could spell trouble for machine learning
Advisers warn of discrepancy in advice to regulators over automated decision-making
Industry participants have voiced concern over an apparent discrepancy between the text of the European Union’s new data privacy laws and subsequent interpretative guidance to regulators on how the new regime should treat a key use case of machine learning. Advisers say the first legal precedents set under the rules could have major implications for banks’ use of machine learning-based algorithms to automate everyday decisions such as credit card approvals.
Article 22 of the General Data Protection Regulation (GDPR), which came into effect on May 25, says a consumer has the right not to be subject to a solely automated decision that “produces legal effects concerning him or her, or similarly significantly affects him or her”.
Exceptions exist, however, where such automation is deemed a necessary part of a contract between two parties, or where explicitly authorised by local law. In this context, industry experts believe the right to a human decision-maker would be an active one that consumers will need to invoke.
However, guidelines on GDPR issued earlier this year by the Article 29 Working Party – a group of representatives from local data-protection regulators of EU member states, now superseded by a permanent body – interpreted Article 22 to be a passive right. This implies solely automated decision-making would be generally prohibited, rather than a right that consumers have to request.
Such a strict reading of the regulation could have a severe impact on banks’ rapidly growing use of machine-learning techniques – a subset of artificial intelligence (AI) that relies on automation to create accurate predictions from large, dense data sets – to speed up decisions, such as whether to pre-approve a potential customer for a loan, experts say. This discrepancy has left market participants uncertain how to comply with the regulation.
“[The guidance] is not clear. Many organisations are battling with it because the regulation says one thing, and then the guidance comes out and it actually doesn’t clarify, but almost provides more confusion over some of these things,” says one senior London-based cyber risk consultant.
Is a staff check of one in a million enough human intervention? Probably not
Cyber risk consultant
The Article 29 Working Party was replaced formally by the European Data Protection Board at the end of May, when the GDPR entered force. The EDPB’s goal is to ensure supervisory authorities interpret and enforce the regulation in a consistent way across the different member states.
Asked for comment, a spokesperson for the EDPB says that, given the sheer breadth of firms affected by the GDPR, offering regulatory guidelines customisable for individual people or firms would have been unfeasible. Instead, where banks and other financial services firms acting as data controllers – defined as any party that determines the purpose and means of processing personal data – feel the regulation should be interpreted and applied differently, they must log this with the regulator and justify why.
“As data protection is a wide field with various different procedures, depending on how the data is processed and used, we cannot provide tailor-made codes of conduct for organisations,” the spokesperson says. “The ultimate responsibility for GDPR compliance lies with the controllers. If they believe the GDPR needs to be implemented in a certain way to suit their specific needs, which may differ from the proposed method by the EDPB, they will need to log and justify this for possible future audits [or] controls.”
Bigger issue
Andrew Burt, chief privacy officer and legal engineer of data-management platform Immuta, agrees that the difference between the base text of the GDPR and February’s guidance appears substantial. However, he says the bigger issue is the uncertainty around how regulators plan to enforce the GDPR on machine learning in general.
“The Article 29 Working Party interpretation would add a significant compliance burden to machine learning within the EU, pure and simple,” Burt says. “What I’m concerned about is the current, pretty significant ambiguity in how data-protection authorities are actually going to implement these provisions. When it comes to automated decision-making, the line between GDPR compliance and non-compliance is simply not yet clear.”
That indecision has left market participants questioning how they should oversee automated and machine-learning techniques within their own firms.
The senior consultant suggests banks could look to implement an “airgap” of human intervention in their models. However, the level of involvement required from humans before a model can be judged as being no longer solely automated is unknown, he adds. In the absence of further guidance, firms will have to rely on legal advice when implementing such measures, which will then be tested by regulators and a precedent set.
“Is a staff check of one in a million enough human intervention? Probably not, but [a precedent will be set] when you go down this untested road of how this is going to fall out in court,” he says. “AI is a massive thing that is not going to be stopped by [the] GDPR – but it certainly will make organisations think about [whether] they do it in the right way, in an ethical and transparent way, and how they do it to the benefit of people and themselves while not harming other people with the decisions being made.”
Editing by Tom Osborn
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@risk.net or view our subscription options here: http://subscriptions.risk.net/subscribe
You are currently unable to print this content. Please contact info@risk.net to find out more.
You are currently unable to copy this content. Please contact info@risk.net to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@risk.net
More on Risk management
Climate stress tests are cold comfort for banks
Flaws in regulators’ methodology for gauging financial impact of climate change undermine transition efforts, argues modelling expert
ECB official open to offering liquidity aid to non-banks
Risk Live: Deputy director doesn’t rule out copying UK plan to extend repo facility to pension funds and life insurers
Banks must loosen up on ChatGPT use – risk chiefs
Risk Live: ’Shadow use’ and inability to attract new hires mean restricting access to GPTs is untenable
Simm casts off Covid pain for $40 billion IM reprieve
Recalibration cuts risk weights in equity and commodities, but some credit exposures double on ABX halt
Rate risk modellers relieved as EU deposits stay sticky
Banks feared retail deposits would be flightier than during previous periods of rate hikes
Rough patch: CrowdStrike sparks an auto-update debate
Automating software updates helps keep hackers at bay but can introduce op risk; banks balance the two
Banks urged to keep regulators in the loop on AI plans for AML
Risk managers advocate five-year strategies and compliance teams’ ownership of the tech they use
Banks urged to boost third-party scrutiny amid AML crackdown
Three US regulators highlight deficiencies in banks’ due diligence on fintech partners