Transparent Data Encryption (TDE) in Apache Ignite

[daradurvs]

Hi Igniters.

I have some user cases where I need fast storage with TDE support.
It is requered for PCI DSS certification.

As far as I know AI doesn't support it.

I looked at other storages.
Many storages support it or are engaged in development this feature.

Cassandra community are working on TDE support.[1]

Oracle support it.[2] Moreover it supports indexing and querying on
encrypted data.

I think it will be very usefull to support TDE by AI.

What do you think? Maybe development is already under way?

[1] https://issues.apache.org/jira/browse/CASSANDRA-9945
[2]
https://docs.oracle.com/cd/B19306_01/network.102/b14268/asotrans.htm#ASOAG600

--
Best Regards, Vyacheslav D.

[daradurvs]

Guys, any thoughts?

2017-06-20 11:02 GMT+03:00 Vyacheslav Daradur <[hidden email]>:

> Hi Igniters.
>
> I have some user cases where I need fast storage with TDE support.
> It is requered for PCI DSS certification.
>
> As far as I know AI doesn't support it.
>
> I looked at other storages.
> Many storages support it or are engaged in development this feature.
>
> Cassandra community are working on TDE support.[1]
>
> Oracle support it.[2] Moreover it supports indexing and querying on
> encrypted data.
>
> I think it will be very usefull to support TDE by AI.
>
> What do you think? Maybe development is already under way?
>
> [1] https://issues.apache.org/jira/browse/CASSANDRA-9945
> [2] https://docs.oracle.com/cd/B19306_01/network.102/b14268/
> asotrans.htm#ASOAG600
>
> --
> Best Regards, Vyacheslav D.
>

--
Best Regards, Vyacheslav D.

[Sergi]

I think no one is interested in this stuff right now.

Also as far as I see TDE is just an option for PCI DSS compliancy but not a
requirement.

Your system should pass PCI DSS if you will do the required encryption at
the application level and will properly manage encryption keys.

Sergi

2017-06-26 11:40 GMT+03:00 Vyacheslav Daradur <[hidden email]>:

> Guys, any thoughts?
>
> 2017-06-20 11:02 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
>
> > Hi Igniters.
> >
> > I have some user cases where I need fast storage with TDE support.
> > It is requered for PCI DSS certification.
> >
> > As far as I know AI doesn't support it.
> >
> > I looked at other storages.
> > Many storages support it or are engaged in development this feature.
> >
> > Cassandra community are working on TDE support.[1]
> >
> > Oracle support it.[2] Moreover it supports indexing and querying on
> > encrypted data.
> >
> > I think it will be very usefull to support TDE by AI.
> >
> > What do you think? Maybe development is already under way?
> >
> > [1] https://issues.apache.org/jira/browse/CASSANDRA-9945
> > [2] https://docs.oracle.com/cd/B19306_01/network.102/b14268/
> > asotrans.htm#ASOAG600
> >
> > --
> > Best Regards, Vyacheslav D.
> >
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[daradurvs]

Sergi, thanks for the answer.

>> see TDE is just an option for PCI DSS compliancy but not a requirement.
Requirement: "Protect stored cardholder data"
Encryption is required.
TDE - is one of ways to implement it at the database level.

Sure, an implementation at the application level solve it.

I meant another.
I thought maybe this feature is in the Ignite roadmap?


2017-06-26 13:53 GMT+03:00 Sergi Vladykin <[hidden email]>:

> I think no one is interested in this stuff right now.
>
> Also as far as I see TDE is just an option for PCI DSS compliancy but not a
> requirement.
>
> Your system should pass PCI DSS if you will do the required encryption at
> the application level and will properly manage encryption keys.
>
> Sergi
>
> 2017-06-26 11:40 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
>
> > Guys, any thoughts?
> >
> > 2017-06-20 11:02 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
> >
> > > Hi Igniters.
> > >
> > > I have some user cases where I need fast storage with TDE support.
> > > It is requered for PCI DSS certification.
> > >
> > > As far as I know AI doesn't support it.
> > >
> > > I looked at other storages.
> > > Many storages support it or are engaged in development this feature.
> > >
> > > Cassandra community are working on TDE support.[1]
> > >
> > > Oracle support it.[2] Moreover it supports indexing and querying on
> > > encrypted data.
> > >
> > > I think it will be very usefull to support TDE by AI.
> > >
> > > What do you think? Maybe development is already under way?
> > >
> > > [1] https://issues.apache.org/jira/browse/CASSANDRA-9945
> > > [2] https://docs.oracle.com/cd/B19306_01/network.102/b14268/
> > > asotrans.htm#ASOAG600
> > >
> > > --
> > > Best Regards, Vyacheslav D.
> > >
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
> >
>

--
Best Regards, Vyacheslav D.

[Sergi]

No, we don't have plans for it.

Sergi

2017-06-26 14:20 GMT+03:00 Vyacheslav Daradur <[hidden email]>:

> Sergi, thanks for the answer.
>
> >> see TDE is just an option for PCI DSS compliancy but not a requirement.
> Requirement: "Protect stored cardholder data"
> Encryption is required.
> TDE - is one of ways to implement it at the database level.
>
> Sure, an implementation at the application level solve it.
>
> I meant another.
> I thought maybe this feature is in the Ignite roadmap?
>
>
> 2017-06-26 13:53 GMT+03:00 Sergi Vladykin <[hidden email]>:
>
> > I think no one is interested in this stuff right now.
> >
> > Also as far as I see TDE is just an option for PCI DSS compliancy but
> not a
> > requirement.
> >
> > Your system should pass PCI DSS if you will do the required encryption at
> > the application level and will properly manage encryption keys.
> >
> > Sergi
> >
> > 2017-06-26 11:40 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
> >
> > > Guys, any thoughts?
> > >
> > > 2017-06-20 11:02 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
> > >
> > > > Hi Igniters.
> > > >
> > > > I have some user cases where I need fast storage with TDE support.
> > > > It is requered for PCI DSS certification.
> > > >
> > > > As far as I know AI doesn't support it.
> > > >
> > > > I looked at other storages.
> > > > Many storages support it or are engaged in development this feature.
> > > >
> > > > Cassandra community are working on TDE support.[1]
> > > >
> > > > Oracle support it.[2] Moreover it supports indexing and querying on
> > > > encrypted data.
> > > >
> > > > I think it will be very usefull to support TDE by AI.
> > > >
> > > > What do you think? Maybe development is already under way?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/CASSANDRA-9945
> > > > [2] https://docs.oracle.com/cd/B19306_01/network.102/b14268/
> > > > asotrans.htm#ASOAG600
> > > >
> > > > --
> > > > Best Regards, Vyacheslav D.
> > > >
> > >
> > >
> > >
> > > --
> > > Best Regards, Vyacheslav D.
> > >
> >
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[Dmitrii Ryabov]

Hi, Igniters! I think it would be nice to implement encryption in
Ignite. Even SQLite and H2 have encryption so why Ignite don't have
it?

I'd like to propose a design for discussion.

Configurations:
IgniteConfiguration:
- KeyStore tdeKeyStore - contain encryption keys.
- Encryptor encryptor - interface for encryption algorythm with
methods `encrypt()`, `decrypt()`.
CacheConfiguration:
- TransparentDataEncryption tde - configures where or how we will
encrypt. We can encrypt pages (PDS or PDS&offheap) or cached objects
(PDS, PDS&offheap and even heap if we encrypt just after key
validation and security check). And, of course, TDE can be turned off.

Encryption keys:
Every node has keystore with Cache Encryption Keys (CEKs), one key for
one cache, they are stored in encrypted form.
CEKs are encrypted by Master Encryption Key (MEK), which stored on the
coordinator or somewhere out of cluster (this way is safer). Also, we
need backups for MEK on other nodes or somewhere out of the cluster.

Encryption:
To make any operation with encrypting cache node gets MEK from
coordinator (or somewhere out of the cluster). Then it decrypts needed
CEK. With decrypted CEK we encrypt/decrypt data. When transaction
finished we must destroy received MEK.

TransparentDataEncryption enum:
NONE - no encryption.
ALL_OBJECTS - data will be encrypted on tx initiator just after key
validation and security check, so most of time it would be just a
byte[] object. It is good, but on the other hand we have additional
encryption for any node with listeners for data changing.
PDS_ONLY_OBJECTS - data will be encrypted on primary and backups when
storing to PDS.
OFFHEAP_PDS_OBJECTS - data will be encrypted when storing to offheap
storage and PDS.
PDS_ONLY_PAGES - data pages will be encrypted on primary and backups
when storing to PDS.
OFFHEAP_PDS_PAGES - data pages will be encrypted when storing to
offheap storage and PDS.

But I'm not sure about objects/pages. Do we really need both of them?


2017-06-26 14:39 GMT+03:00 Sergi Vladykin <[hidden email]>:

> No, we don't have plans for it.
>
> Sergi
>
> 2017-06-26 14:20 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
>
>> Sergi, thanks for the answer.
>>
>> >> see TDE is just an option for PCI DSS compliancy but not a requirement.
>> Requirement: "Protect stored cardholder data"
>> Encryption is required.
>> TDE - is one of ways to implement it at the database level.
>>
>> Sure, an implementation at the application level solve it.
>>
>> I meant another.
>> I thought maybe this feature is in the Ignite roadmap?
>>
>>
>> 2017-06-26 13:53 GMT+03:00 Sergi Vladykin <[hidden email]>:
>>
>> > I think no one is interested in this stuff right now.
>> >
>> > Also as far as I see TDE is just an option for PCI DSS compliancy but
>> not a
>> > requirement.
>> >
>> > Your system should pass PCI DSS if you will do the required encryption at
>> > the application level and will properly manage encryption keys.
>> >
>> > Sergi
>> >
>> > 2017-06-26 11:40 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
>> >
>> > > Guys, any thoughts?
>> > >
>> > > 2017-06-20 11:02 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
>> > >
>> > > > Hi Igniters.
>> > > >
>> > > > I have some user cases where I need fast storage with TDE support.
>> > > > It is requered for PCI DSS certification.
>> > > >
>> > > > As far as I know AI doesn't support it.
>> > > >
>> > > > I looked at other storages.
>> > > > Many storages support it or are engaged in development this feature.
>> > > >
>> > > > Cassandra community are working on TDE support.[1]
>> > > >
>> > > > Oracle support it.[2] Moreover it supports indexing and querying on
>> > > > encrypted data.
>> > > >
>> > > > I think it will be very usefull to support TDE by AI.
>> > > >
>> > > > What do you think? Maybe development is already under way?
>> > > >
>> > > > [1] https://issues.apache.org/jira/browse/CASSANDRA-9945
>> > > > [2] https://docs.oracle.com/cd/B19306_01/network.102/b14268/
>> > > > asotrans.htm#ASOAG600
>> > > >
>> > > > --
>> > > > Best Regards, Vyacheslav D.
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Best Regards, Vyacheslav D.
>> > >
>> >
>>
>>
>>
>> --
>> Best Regards, Vyacheslav D.
>>

[Dmitriy Pavlov]

Hi Igniters,

Encryption will ensure security only if keys are stored in a secure
storage. Otherwise having access to Ignite node storage, we can extract
encryption master keys values and read data as plain text (page's clear
content).

Where are we going to store keys (MEK) physically? Whould it be PKCS#11
storage? Where we will store passwords to unlock storage or it will be
responibilty of user?

One more thing is how "node gets MEK from coordinator", if we send
cleartext MEK, such security becomes useless also.

Encryption will have negative influence to performance, so I think we
shouldn't do this thing as first priority. And we need to clearly realize
whole security scheme.

Sincerely,
Dmitriy Pavlov

пн, 5 февр. 2018 г. в 11:23, Дмитрий Рябов <[hidden email]>:

> Hi, Igniters! I think it would be nice to implement encryption in
> Ignite. Even SQLite and H2 have encryption so why Ignite don't have
> it?
>
> I'd like to propose a design for discussion.
>
> Configurations:
> IgniteConfiguration:
> - KeyStore tdeKeyStore - contain encryption keys.
> - Encryptor encryptor - interface for encryption algorythm with
> methods `encrypt()`, `decrypt()`.
> CacheConfiguration:
> - TransparentDataEncryption tde - configures where or how we will
> encrypt. We can encrypt pages (PDS or PDS&offheap) or cached objects
> (PDS, PDS&offheap and even heap if we encrypt just after key
> validation and security check). And, of course, TDE can be turned off.
>
> Encryption keys:
> Every node has keystore with Cache Encryption Keys (CEKs), one key for
> one cache, they are stored in encrypted form.
> CEKs are encrypted by Master Encryption Key (MEK), which stored on the
> coordinator or somewhere out of cluster (this way is safer). Also, we
> need backups for MEK on other nodes or somewhere out of the cluster.
>
> Encryption:
> To make any operation with encrypting cache node gets MEK from
> coordinator (or somewhere out of the cluster). Then it decrypts needed
> CEK. With decrypted CEK we encrypt/decrypt data. When transaction
> finished we must destroy received MEK.
>
> TransparentDataEncryption enum:
> NONE - no encryption.
> ALL_OBJECTS - data will be encrypted on tx initiator just after key
> validation and security check, so most of time it would be just a
> byte[] object. It is good, but on the other hand we have additional
> encryption for any node with listeners for data changing.
> PDS_ONLY_OBJECTS - data will be encrypted on primary and backups when
> storing to PDS.
> OFFHEAP_PDS_OBJECTS - data will be encrypted when storing to offheap
> storage and PDS.
> PDS_ONLY_PAGES - data pages will be encrypted on primary and backups
> when storing to PDS.
> OFFHEAP_PDS_PAGES - data pages will be encrypted when storing to
> offheap storage and PDS.
>
> But I'm not sure about objects/pages. Do we really need both of them?
>
>
> 2017-06-26 14:39 GMT+03:00 Sergi Vladykin <[hidden email]>:
> > No, we don't have plans for it.
> >
> > Sergi
> >
> > 2017-06-26 14:20 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
> >
> >> Sergi, thanks for the answer.
> >>
> >> >> see TDE is just an option for PCI DSS compliancy but not a
> requirement.
> >> Requirement: "Protect stored cardholder data"
> >> Encryption is required.
> >> TDE - is one of ways to implement it at the database level.
> >>
> >> Sure, an implementation at the application level solve it.
> >>
> >> I meant another.
> >> I thought maybe this feature is in the Ignite roadmap?
> >>
> >>
> >> 2017-06-26 13:53 GMT+03:00 Sergi Vladykin <[hidden email]>:
> >>
> >> > I think no one is interested in this stuff right now.
> >> >
> >> > Also as far as I see TDE is just an option for PCI DSS compliancy but
> >> not a
> >> > requirement.
> >> >
> >> > Your system should pass PCI DSS if you will do the required
> encryption at
> >> > the application level and will properly manage encryption keys.
> >> >
> >> > Sergi
> >> >
> >> > 2017-06-26 11:40 GMT+03:00 Vyacheslav Daradur <[hidden email]>:
> >> >
> >> > > Guys, any thoughts?
> >> > >
> >> > > 2017-06-20 11:02 GMT+03:00 Vyacheslav Daradur <[hidden email]
> >:
> >> > >
> >> > > > Hi Igniters.
> >> > > >
> >> > > > I have some user cases where I need fast storage with TDE support.
> >> > > > It is requered for PCI DSS certification.
> >> > > >
> >> > > > As far as I know AI doesn't support it.
> >> > > >
> >> > > > I looked at other storages.
> >> > > > Many storages support it or are engaged in development this
> feature.
> >> > > >
> >> > > > Cassandra community are working on TDE support.[1]
> >> > > >
> >> > > > Oracle support it.[2] Moreover it supports indexing and querying
> on
> >> > > > encrypted data.
> >> > > >
> >> > > > I think it will be very usefull to support TDE by AI.
> >> > > >
> >> > > > What do you think? Maybe development is already under way?
> >> > > >
> >> > > > [1] https://issues.apache.org/jira/browse/CASSANDRA-9945
> >> > > > [2] https://docs.oracle.com/cd/B19306_01/network.102/b14268/
> >> > > > asotrans.htm#ASOAG600
> >> > > >
> >> > > > --
> >> > > > Best Regards, Vyacheslav D.
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Best Regards, Vyacheslav D.
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Best Regards, Vyacheslav D.
> >>
>

[Dmitrii Ryabov]

Hello, Igniters!

I investigated the issue and wrote some details in a draft document
[1]. I think we should made IEP for TDE because it is a big change and
should be described in a single place, but not in a message
conversation.
Please, look it and write your thoughts. What is not understandable,
what should be detailed or described?

> Where are we going to store keys (MEK) physically? Would it be PKCS#11
> storage? Where we will store passwords to unlock storage or it will be
> responibilty of user?

I think we should provide interface for MEK storage to let users use
storages they want. I suppose at the first step we should provide very
simple implementation, which will store MEK on every node and MEK will
be extracted by administrator during cluster activation process. Once
MEK is extracted from key store, we decrypt CEKs and destroy open MEK,
leaving open only cache keys.

I think external storage is user's worry and we shouldn't give users
built-in external storage like Oracle Wallet or Microsoft Azure Key
Vault because it will increase Ignite's complexity too much.

And yes, we should to comply with the standards like PKCS#11.

> One more thing is how "node gets MEK from coordinator", if we send
> cleartext MEK, such security becomes useless also.

Yeah, that's why we should use secured connection. As I know, we have
SSL implementation over JDK implementation, am I right? But we must
ensure to use latest SSL/TLS version.

[1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf

[Nikolay Izhikov-2]

Hell, Dima!

Thank you for document!

I'm ready to implement this feature with you.

Igniters, please, share you thoughts about proposed design

[1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf

В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:

> Hello, Igniters!
>
> I investigated the issue and wrote some details in a draft document
> [1]. I think we should made IEP for TDE because it is a big change and
> should be described in a single place, but not in a message
> conversation.
> Please, look it and write your thoughts. What is not understandable,
> what should be detailed or described?
>
> > Where are we going to store keys (MEK) physically? Would it be PKCS#11
> > storage? Where we will store passwords to unlock storage or it will be
> > responibilty of user?
>
> I think we should provide interface for MEK storage to let users use
> storages they want. I suppose at the first step we should provide very
> simple implementation, which will store MEK on every node and MEK will
> be extracted by administrator during cluster activation process. Once
> MEK is extracted from key store, we decrypt CEKs and destroy open MEK,
> leaving open only cache keys.
>
> I think external storage is user's worry and we shouldn't give users
> built-in external storage like Oracle Wallet or Microsoft Azure Key
> Vault because it will increase Ignite's complexity too much.
>
> And yes, we should to comply with the standards like PKCS#11.
>
> > One more thing is how "node gets MEK from coordinator", if we send
> > cleartext MEK, such security becomes useless also.
>
> Yeah, that's why we should use secured connection. As I know, we have
> SSL implementation over JDK implementation, am I right? But we must
> ensure to use latest SSL/TLS version.
>
> [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf

[dsetrakyan]

I have looked at the design, but could not find anything about running SQL
queries against the encrypted data. Will it be supported?

D.

On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <[hidden email]> wrote:

> Hell, Dima!
>
> Thank you for document!
>
> I'm ready to implement this feature with you.
>
> Igniters, please, share you thoughts about proposed design
>
> [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>
> В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > Hello, Igniters!
> >
> > I investigated the issue and wrote some details in a draft document
> > [1]. I think we should made IEP for TDE because it is a big change and
> > should be described in a single place, but not in a message
> > conversation.
> > Please, look it and write your thoughts. What is not understandable,
> > what should be detailed or described?
> >
> > > Where are we going to store keys (MEK) physically? Would it be PKCS#11
> > > storage? Where we will store passwords to unlock storage or it will be
> > > responibilty of user?
> >
> > I think we should provide interface for MEK storage to let users use
> > storages they want. I suppose at the first step we should provide very
> > simple implementation, which will store MEK on every node and MEK will
> > be extracted by administrator during cluster activation process. Once
> > MEK is extracted from key store, we decrypt CEKs and destroy open MEK,
> > leaving open only cache keys.
> >
> > I think external storage is user's worry and we shouldn't give users
> > built-in external storage like Oracle Wallet or Microsoft Azure Key
> > Vault because it will increase Ignite's complexity too much.
> >
> > And yes, we should to comply with the standards like PKCS#11.
> >
> > > One more thing is how "node gets MEK from coordinator", if we send
> > > cleartext MEK, such security becomes useless also.
> >
> > Yeah, that's why we should use secured connection. As I know, we have
> > SSL implementation over JDK implementation, am I right? But we must
> > ensure to use latest SSL/TLS version.
> >
> > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>

[Nikolay Izhikov-2]

Hello, Dmitriy.

Thank you for feedback!

> Will it be supported?

Yes.

TDE shouldn't broke any of existing Ignite features.
It adds some encrypt/decrypt level when we writing and reading pages in/from PDS.

В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:

> I have looked at the design, but could not find anything about running SQL
> queries against the encrypted data. Will it be supported?
>
> D.
>
> On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <[hidden email]> wrote:
>
> > Hell, Dima!
> >
> > Thank you for document!
> >
> > I'm ready to implement this feature with you.
> >
> > Igniters, please, share you thoughts about proposed design
> >
> > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> >
> > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > > Hello, Igniters!
> > >
> > > I investigated the issue and wrote some details in a draft document
> > > [1]. I think we should made IEP for TDE because it is a big change and
> > > should be described in a single place, but not in a message
> > > conversation.
> > > Please, look it and write your thoughts. What is not understandable,
> > > what should be detailed or described?
> > >
> > > > Where are we going to store keys (MEK) physically? Would it be PKCS#11
> > > > storage? Where we will store passwords to unlock storage or it will be
> > > > responibilty of user?
> > >
> > > I think we should provide interface for MEK storage to let users use
> > > storages they want. I suppose at the first step we should provide very
> > > simple implementation, which will store MEK on every node and MEK will
> > > be extracted by administrator during cluster activation process. Once
> > > MEK is extracted from key store, we decrypt CEKs and destroy open MEK,
> > > leaving open only cache keys.
> > >
> > > I think external storage is user's worry and we shouldn't give users
> > > built-in external storage like Oracle Wallet or Microsoft Azure Key
> > > Vault because it will increase Ignite's complexity too much.
> > >
> > > And yes, we should to comply with the standards like PKCS#11.
> > >
> > > > One more thing is how "node gets MEK from coordinator", if we send
> > > > cleartext MEK, such security becomes useless also.
> > >
> > > Yeah, that's why we should use secured connection. As I know, we have
> > > SSL implementation over JDK implementation, am I right? But we must
> > > ensure to use latest SSL/TLS version.
> > >
> > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf

[dsetrakyan]

On Fri, Mar 2, 2018 at 8:29 AM, Nikolay Izhikov <[hidden email]> wrote:

> Hello, Dmitriy.
>
> Thank you for feedback!
>
> > Will it be supported?
>
> Yes.
>
> TDE shouldn't broke any of existing Ignite features.
> It adds some encrypt/decrypt level when we writing and reading pages
> in/from PDS.
>

Got it, thanks! In this case this sounds very useful. Do we understand the
performance impact?

[Nikolay Izhikov-2]

> Got it, thanks! In this case this sounds very useful. Do we understand the performance impact?

I don't think we fully understand it.
It's a question of additional research and benchmarking.

So preliminary conclusions, based on common sense and my experiense of usage Ignite in production,  are following:

1. CPU is not a bottleneck in most of Ignite use-cases.
Especially when we use it as a in-memory database.

2. TDE mostly will increase CPU usage: SSL + pages encryption/decryption.

В Пт, 02/03/2018 в 08:33 +0300, Dmitriy Setrakyan пишет:

> On Fri, Mar 2, 2018 at 8:29 AM, Nikolay Izhikov <[hidden email]> wrote:
>
> > Hello, Dmitriy.
> >
> > Thank you for feedback!
> >
> > > Will it be supported?
> >
> > Yes.
> >
> > TDE shouldn't broke any of existing Ignite features.
> > It adds some encrypt/decrypt level when we writing and reading pages
> > in/from PDS.
> >
>
> Got it, thanks! In this case this sounds very useful. Do we understand the
> performance impact?

[dmagda]

Dmitriy R., Nilokay,

Thanks for the analysis and handout of the architectural design. No doubts,
it would be a valuable addition to Ignite.

I would encourage you creating an IEP on the wiki and break the work into
pieces discussing specific part with the community.

--
Denis


On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <[hidden email]> wrote:

> Hello, Dmitriy.
>
> Thank you for feedback!
>
> > Will it be supported?
>
> Yes.
>
> TDE shouldn't broke any of existing Ignite features.
> It adds some encrypt/decrypt level when we writing and reading pages
> in/from PDS.
>
> В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > I have looked at the design, but could not find anything about running
> SQL
> > queries against the encrypted data. Will it be supported?
> >
> > D.
> >
> > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <[hidden email]>
> wrote:
> >
> > > Hell, Dima!
> > >
> > > Thank you for document!
> > >
> > > I'm ready to implement this feature with you.
> > >
> > > Igniters, please, share you thoughts about proposed design
> > >
> > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > >
> > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > > > Hello, Igniters!
> > > >
> > > > I investigated the issue and wrote some details in a draft document
> > > > [1]. I think we should made IEP for TDE because it is a big change
> and
> > > > should be described in a single place, but not in a message
> > > > conversation.
> > > > Please, look it and write your thoughts. What is not understandable,
> > > > what should be detailed or described?
> > > >
> > > > > Where are we going to store keys (MEK) physically? Would it be
> PKCS#11
> > > > > storage? Where we will store passwords to unlock storage or it
> will be
> > > > > responibilty of user?
> > > >
> > > > I think we should provide interface for MEK storage to let users use
> > > > storages they want. I suppose at the first step we should provide
> very
> > > > simple implementation, which will store MEK on every node and MEK
> will
> > > > be extracted by administrator during cluster activation process. Once
> > > > MEK is extracted from key store, we decrypt CEKs and destroy open
> MEK,
> > > > leaving open only cache keys.
> > > >
> > > > I think external storage is user's worry and we shouldn't give users
> > > > built-in external storage like Oracle Wallet or Microsoft Azure Key
> > > > Vault because it will increase Ignite's complexity too much.
> > > >
> > > > And yes, we should to comply with the standards like PKCS#11.
> > > >
> > > > > One more thing is how "node gets MEK from coordinator", if we send
> > > > > cleartext MEK, such security becomes useless also.
> > > >
> > > > Yeah, that's why we should use secured connection. As I know, we have
> > > > SSL implementation over JDK implementation, am I right? But we must
> > > > ensure to use latest SSL/TLS version.
> > > >
> > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>

[daradurvs]

Dima, great job!

Looking forward to the feature completion!

On Fri, Mar 2, 2018 at 9:23 AM, Denis Magda <[hidden email]> wrote:

> Dmitriy R., Nilokay,
>
> Thanks for the analysis and handout of the architectural design. No doubts,
> it would be a valuable addition to Ignite.
>
> I would encourage you creating an IEP on the wiki and break the work into
> pieces discussing specific part with the community.
>
> --
> Denis
>
>
> On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <[hidden email]> wrote:
>
>> Hello, Dmitriy.
>>
>> Thank you for feedback!
>>
>> > Will it be supported?
>>
>> Yes.
>>
>> TDE shouldn't broke any of existing Ignite features.
>> It adds some encrypt/decrypt level when we writing and reading pages
>> in/from PDS.
>>
>> В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
>> > I have looked at the design, but could not find anything about running
>> SQL
>> > queries against the encrypted data. Will it be supported?
>> >
>> > D.
>> >
>> > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <[hidden email]>
>> wrote:
>> >
>> > > Hell, Dima!
>> > >
>> > > Thank you for document!
>> > >
>> > > I'm ready to implement this feature with you.
>> > >
>> > > Igniters, please, share you thoughts about proposed design
>> > >
>> > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>> > >
>> > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
>> > > > Hello, Igniters!
>> > > >
>> > > > I investigated the issue and wrote some details in a draft document
>> > > > [1]. I think we should made IEP for TDE because it is a big change
>> and
>> > > > should be described in a single place, but not in a message
>> > > > conversation.
>> > > > Please, look it and write your thoughts. What is not understandable,
>> > > > what should be detailed or described?
>> > > >
>> > > > > Where are we going to store keys (MEK) physically? Would it be
>> PKCS#11
>> > > > > storage? Where we will store passwords to unlock storage or it
>> will be
>> > > > > responibilty of user?
>> > > >
>> > > > I think we should provide interface for MEK storage to let users use
>> > > > storages they want. I suppose at the first step we should provide
>> very
>> > > > simple implementation, which will store MEK on every node and MEK
>> will
>> > > > be extracted by administrator during cluster activation process. Once
>> > > > MEK is extracted from key store, we decrypt CEKs and destroy open
>> MEK,
>> > > > leaving open only cache keys.
>> > > >
>> > > > I think external storage is user's worry and we shouldn't give users
>> > > > built-in external storage like Oracle Wallet or Microsoft Azure Key
>> > > > Vault because it will increase Ignite's complexity too much.
>> > > >
>> > > > And yes, we should to comply with the standards like PKCS#11.
>> > > >
>> > > > > One more thing is how "node gets MEK from coordinator", if we send
>> > > > > cleartext MEK, such security becomes useless also.
>> > > >
>> > > > Yeah, that's why we should use secured connection. As I know, we have
>> > > > SSL implementation over JDK implementation, am I right? But we must
>> > > > ensure to use latest SSL/TLS version.
>> > > >
>> > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>>

--
Best Regards, Vyacheslav D.

[Nikolay Izhikov-2]

Hello, Denis.

> I would encourage you creating an IEP

That is exactly what we want to do :)

But seems I have not sufficient privileges to do it on Ignite wiki.

https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals

Can you or someone give me such rights?

В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:

> Dmitriy R., Nilokay,
>
> Thanks for the analysis and handout of the architectural design. No doubts,
> it would be a valuable addition to Ignite.
>
> I would encourage you creating an IEP on the wiki and break the work into
> pieces discussing specific part with the community.
>
> --
> Denis
>
>
> On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <[hidden email]> wrote:
>
> > Hello, Dmitriy.
> >
> > Thank you for feedback!
> >
> > > Will it be supported?
> >
> > Yes.
> >
> > TDE shouldn't broke any of existing Ignite features.
> > It adds some encrypt/decrypt level when we writing and reading pages
> > in/from PDS.
> >
> > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > > I have looked at the design, but could not find anything about running
> >
> > SQL
> > > queries against the encrypted data. Will it be supported?
> > >
> > > D.
> > >
> > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <[hidden email]>
> >
> > wrote:
> > >
> > > > Hell, Dima!
> > > >
> > > > Thank you for document!
> > > >
> > > > I'm ready to implement this feature with you.
> > > >
> > > > Igniters, please, share you thoughts about proposed design
> > > >
> > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > > >
> > > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > > > > Hello, Igniters!
> > > > >
> > > > > I investigated the issue and wrote some details in a draft document
> > > > > [1]. I think we should made IEP for TDE because it is a big change
> >
> > and
> > > > > should be described in a single place, but not in a message
> > > > > conversation.
> > > > > Please, look it and write your thoughts. What is not understandable,
> > > > > what should be detailed or described?
> > > > >
> > > > > > Where are we going to store keys (MEK) physically? Would it be
> >
> > PKCS#11
> > > > > > storage? Where we will store passwords to unlock storage or it
> >
> > will be
> > > > > > responibilty of user?
> > > > >
> > > > > I think we should provide interface for MEK storage to let users use
> > > > > storages they want. I suppose at the first step we should provide
> >
> > very
> > > > > simple implementation, which will store MEK on every node and MEK
> >
> > will
> > > > > be extracted by administrator during cluster activation process. Once
> > > > > MEK is extracted from key store, we decrypt CEKs and destroy open
> >
> > MEK,
> > > > > leaving open only cache keys.
> > > > >
> > > > > I think external storage is user's worry and we shouldn't give users
> > > > > built-in external storage like Oracle Wallet or Microsoft Azure Key
> > > > > Vault because it will increase Ignite's complexity too much.
> > > > >
> > > > > And yes, we should to comply with the standards like PKCS#11.
> > > > >
> > > > > > One more thing is how "node gets MEK from coordinator", if we send
> > > > > > cleartext MEK, such security becomes useless also.
> > > > >
> > > > > Yeah, that's why we should use secured connection. As I know, we have
> > > > > SSL implementation over JDK implementation, am I right? But we must
> > > > > ensure to use latest SSL/TLS version.
> > > > >
> > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf

[Denis Magda]

Nikolay, what's your Wiki ID? I'll grant you required permissions.

--
Denis

On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov <[hidden email]>
wrote:

> Hello, Denis.
>
> > I would encourage you creating an IEP
>
> That is exactly what we want to do :)
>
> But seems I have not sufficient privileges to do it on Ignite wiki.
>
> https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
>
> Can you or someone give me such rights?
>
> В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
> > Dmitriy R., Nilokay,
> >
> > Thanks for the analysis and handout of the architectural design. No
> doubts,
> > it would be a valuable addition to Ignite.
> >
> > I would encourage you creating an IEP on the wiki and break the work into
> > pieces discussing specific part with the community.
> >
> > --
> > Denis
> >
> >
> > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <[hidden email]>
> wrote:
> >
> > > Hello, Dmitriy.
> > >
> > > Thank you for feedback!
> > >
> > > > Will it be supported?
> > >
> > > Yes.
> > >
> > > TDE shouldn't broke any of existing Ignite features.
> > > It adds some encrypt/decrypt level when we writing and reading pages
> > > in/from PDS.
> > >
> > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > > > I have looked at the design, but could not find anything about
> running
> > >
> > > SQL
> > > > queries against the encrypted data. Will it be supported?
> > > >
> > > > D.
> > > >
> > > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <[hidden email]
> >
> > >
> > > wrote:
> > > >
> > > > > Hell, Dima!
> > > > >
> > > > > Thank you for document!
> > > > >
> > > > > I'm ready to implement this feature with you.
> > > > >
> > > > > Igniters, please, share you thoughts about proposed design
> > > > >
> > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > > > >
> > > > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > > > > > Hello, Igniters!
> > > > > >
> > > > > > I investigated the issue and wrote some details in a draft
> document
> > > > > > [1]. I think we should made IEP for TDE because it is a big
> change
> > >
> > > and
> > > > > > should be described in a single place, but not in a message
> > > > > > conversation.
> > > > > > Please, look it and write your thoughts. What is not
> understandable,
> > > > > > what should be detailed or described?
> > > > > >
> > > > > > > Where are we going to store keys (MEK) physically? Would it be
> > >
> > > PKCS#11
> > > > > > > storage? Where we will store passwords to unlock storage or it
> > >
> > > will be
> > > > > > > responibilty of user?
> > > > > >
> > > > > > I think we should provide interface for MEK storage to let users
> use
> > > > > > storages they want. I suppose at the first step we should provide
> > >
> > > very
> > > > > > simple implementation, which will store MEK on every node and MEK
> > >
> > > will
> > > > > > be extracted by administrator during cluster activation process.
> Once
> > > > > > MEK is extracted from key store, we decrypt CEKs and destroy open
> > >
> > > MEK,
> > > > > > leaving open only cache keys.
> > > > > >
> > > > > > I think external storage is user's worry and we shouldn't give
> users
> > > > > > built-in external storage like Oracle Wallet or Microsoft Azure
> Key
> > > > > > Vault because it will increase Ignite's complexity too much.
> > > > > >
> > > > > > And yes, we should to comply with the standards like PKCS#11.
> > > > > >
> > > > > > > One more thing is how "node gets MEK from coordinator", if we
> send
> > > > > > > cleartext MEK, such security becomes useless also.
> > > > > >
> > > > > > Yeah, that's why we should use secured connection. As I know, we
> have
> > > > > > SSL implementation over JDK implementation, am I right? But we
> must
> > > > > > ensure to use latest SSL/TLS version.
> > > > > >
> > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>

[Nikolay Izhikov-2]

Thank you, it's - nizhikov

В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:

> Nikolay, what's your Wiki ID? I'll grant you required permissions.
>
> --
> Denis
>
> On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov <[hidden email]> wrote:
> > Hello, Denis.
> >
> > > I would encourage you creating an IEP
> >
> > That is exactly what we want to do :)
> >
> > But seems I have not sufficient privileges to do it on Ignite wiki.
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
> >
> > Can you or someone give me such rights?
> >
> > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
> > > Dmitriy R., Nilokay,
> > >
> > > Thanks for the analysis and handout of the architectural design. No doubts,
> > > it would be a valuable addition to Ignite.
> > >
> > > I would encourage you creating an IEP on the wiki and break the work into
> > > pieces discussing specific part with the community.
> > >
> > > --
> > > Denis
> > >
> > >
> > > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <[hidden email]> wrote:
> > >
> > > > Hello, Dmitriy.
> > > >
> > > > Thank you for feedback!
> > > >
> > > > > Will it be supported?
> > > >
> > > > Yes.
> > > >
> > > > TDE shouldn't broke any of existing Ignite features.
> > > > It adds some encrypt/decrypt level when we writing and reading pages
> > > > in/from PDS.
> > > >
> > > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > > > > I have looked at the design, but could not find anything about running
> > > >
> > > > SQL
> > > > > queries against the encrypted data. Will it be supported?
> > > > >
> > > > > D.
> > > > >
> > > > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <[hidden email]>
> > > >
> > > > wrote:
> > > > >
> > > > > > Hell, Dima!
> > > > > >
> > > > > > Thank you for document!
> > > > > >
> > > > > > I'm ready to implement this feature with you.
> > > > > >
> > > > > > Igniters, please, share you thoughts about proposed design
> > > > > >
> > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > > > > >
> > > > > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > > > > > > Hello, Igniters!
> > > > > > >
> > > > > > > I investigated the issue and wrote some details in a draft document
> > > > > > > [1]. I think we should made IEP for TDE because it is a big change
> > > >
> > > > and
> > > > > > > should be described in a single place, but not in a message
> > > > > > > conversation.
> > > > > > > Please, look it and write your thoughts. What is not understandable,
> > > > > > > what should be detailed or described?
> > > > > > >
> > > > > > > > Where are we going to store keys (MEK) physically? Would it be
> > > >
> > > > PKCS#11
> > > > > > > > storage? Where we will store passwords to unlock storage or it
> > > >
> > > > will be
> > > > > > > > responibilty of user?
> > > > > > >
> > > > > > > I think we should provide interface for MEK storage to let users use
> > > > > > > storages they want. I suppose at the first step we should provide
> > > >
> > > > very
> > > > > > > simple implementation, which will store MEK on every node and MEK
> > > >
> > > > will
> > > > > > > be extracted by administrator during cluster activation process. Once
> > > > > > > MEK is extracted from key store, we decrypt CEKs and destroy open
> > > >
> > > > MEK,
> > > > > > > leaving open only cache keys.
> > > > > > >
> > > > > > > I think external storage is user's worry and we shouldn't give users
> > > > > > > built-in external storage like Oracle Wallet or Microsoft Azure Key
> > > > > > > Vault because it will increase Ignite's complexity too much.
> > > > > > >
> > > > > > > And yes, we should to comply with the standards like PKCS#11.
> > > > > > >
> > > > > > > > One more thing is how "node gets MEK from coordinator", if we send
> > > > > > > > cleartext MEK, such security becomes useless also.
> > > > > > >
> > > > > > > Yeah, that's why we should use secured connection. As I know, we have
> > > > > > > SSL implementation over JDK implementation, am I right? But we must
> > > > > > > ensure to use latest SSL/TLS version.
> > > > > > >
> > > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>
>

[Alexey Goncharuk]

Guys,

I think this TDE proposal is not thought through enough yet. Please
consider the following points when writing the IEP:

 * How encryption keys will be stored and accessed? If the encryption key
is stored with the same permissions as the main data storage, the whole
exercise with encryption is self-deception
 * Will pages be encrypted on disk or also in memory?
 * How do you make sure that encrypted page fits the page size (I am not an
expert in encryption, so I am not sure if it adds an overhead)
 * As Dmitriy Pavlov mentioned, currently data and index pages are highly
redundant and some of the fields in certain pages are known in advance.
This significantly increases success of known-plain-text attacks. How are
you planning to fix it?
 * How will you write WAL delta records for encrypted pages? If a change in
a single byte will potentially change the whole page, this will induce a
huge write amplification on WAL. How do you encrypt WAL data records? How
will this work with this optimization: [1]

[1] https://ggsystems.atlassian.net/browse/IGN-7789

--AG

2018-03-06 8:55 GMT+03:00 Nikolay Izhikov <[hidden email]>:

> Thank you, it's - nizhikov
>
> В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:
> > Nikolay, what's your Wiki ID? I'll grant you required permissions.
> >
> > --
> > Denis
> >
> > On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov <[hidden email]>
> wrote:
> > > Hello, Denis.
> > >
> > > > I would encourage you creating an IEP
> > >
> > > That is exactly what we want to do :)
> > >
> > > But seems I have not sufficient privileges to do it on Ignite wiki.
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
> > >
> > > Can you or someone give me such rights?
> > >
> > > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
> > > > Dmitriy R., Nilokay,
> > > >
> > > > Thanks for the analysis and handout of the architectural design. No
> doubts,
> > > > it would be a valuable addition to Ignite.
> > > >
> > > > I would encourage you creating an IEP on the wiki and break the work
> into
> > > > pieces discussing specific part with the community.
> > > >
> > > > --
> > > > Denis
> > > >
> > > >
> > > > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <[hidden email]>
> wrote:
> > > >
> > > > > Hello, Dmitriy.
> > > > >
> > > > > Thank you for feedback!
> > > > >
> > > > > > Will it be supported?
> > > > >
> > > > > Yes.
> > > > >
> > > > > TDE shouldn't broke any of existing Ignite features.
> > > > > It adds some encrypt/decrypt level when we writing and reading
> pages
> > > > > in/from PDS.
> > > > >
> > > > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
> > > > > > I have looked at the design, but could not find anything about
> running
> > > > >
> > > > > SQL
> > > > > > queries against the encrypted data. Will it be supported?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <
> [hidden email]>
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > > Hell, Dima!
> > > > > > >
> > > > > > > Thank you for document!
> > > > > > >
> > > > > > > I'm ready to implement this feature with you.
> > > > > > >
> > > > > > > Igniters, please, share you thoughts about proposed design
> > > > > > >
> > > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> > > > > > >
> > > > > > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
> > > > > > > > Hello, Igniters!
> > > > > > > >
> > > > > > > > I investigated the issue and wrote some details in a draft
> document
> > > > > > > > [1]. I think we should made IEP for TDE because it is a big
> change
> > > > >
> > > > > and
> > > > > > > > should be described in a single place, but not in a message
> > > > > > > > conversation.
> > > > > > > > Please, look it and write your thoughts. What is not
> understandable,
> > > > > > > > what should be detailed or described?
> > > > > > > >
> > > > > > > > > Where are we going to store keys (MEK) physically? Would
> it be
> > > > >
> > > > > PKCS#11
> > > > > > > > > storage? Where we will store passwords to unlock storage
> or it
> > > > >
> > > > > will be
> > > > > > > > > responibilty of user?
> > > > > > > >
> > > > > > > > I think we should provide interface for MEK storage to let
> users use
> > > > > > > > storages they want. I suppose at the first step we should
> provide
> > > > >
> > > > > very
> > > > > > > > simple implementation, which will store MEK on every node
> and MEK
> > > > >
> > > > > will
> > > > > > > > be extracted by administrator during cluster activation
> process. Once
> > > > > > > > MEK is extracted from key store, we decrypt CEKs and destroy
> open
> > > > >
> > > > > MEK,
> > > > > > > > leaving open only cache keys.
> > > > > > > >
> > > > > > > > I think external storage is user's worry and we shouldn't
> give users
> > > > > > > > built-in external storage like Oracle Wallet or Microsoft
> Azure Key
> > > > > > > > Vault because it will increase Ignite's complexity too much.
> > > > > > > >
> > > > > > > > And yes, we should to comply with the standards like PKCS#11.
> > > > > > > >
> > > > > > > > > One more thing is how "node gets MEK from coordinator", if
> we send
> > > > > > > > > cleartext MEK, such security becomes useless also.
> > > > > > > >
> > > > > > > > Yeah, that's why we should use secured connection. As I
> know, we have
> > > > > > > > SSL implementation over JDK implementation, am I right? But
> we must
> > > > > > > > ensure to use latest SSL/TLS version.
> > > > > > > >
> > > > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
> >
> >
>

[Alexey Goncharuk]

My bad, the correct link is
https://issues.apache.org/jira/browse/IGNITE-5829

2018-03-06 13:04 GMT+03:00 Alexey Goncharuk <[hidden email]>:

> Guys,
>
> I think this TDE proposal is not thought through enough yet. Please
> consider the following points when writing the IEP:
>
>  * How encryption keys will be stored and accessed? If the encryption key
> is stored with the same permissions as the main data storage, the whole
> exercise with encryption is self-deception
>  * Will pages be encrypted on disk or also in memory?
>  * How do you make sure that encrypted page fits the page size (I am not
> an expert in encryption, so I am not sure if it adds an overhead)
>  * As Dmitriy Pavlov mentioned, currently data and index pages are highly
> redundant and some of the fields in certain pages are known in advance.
> This significantly increases success of known-plain-text attacks. How are
> you planning to fix it?
>  * How will you write WAL delta records for encrypted pages? If a change
> in a single byte will potentially change the whole page, this will induce a
> huge write amplification on WAL. How do you encrypt WAL data records? How
> will this work with this optimization: [1]
>
> [1] https://ggsystems.atlassian.net/browse/IGN-7789
>
> --AG
>
> 2018-03-06 8:55 GMT+03:00 Nikolay Izhikov <[hidden email]>:
>
>> Thank you, it's - nizhikov
>>
>> В Пн, 05/03/2018 в 15:09 -0800, Denis Magda пишет:
>> > Nikolay, what's your Wiki ID? I'll grant you required permissions.
>> >
>> > --
>> > Denis
>> >
>> > On Sun, Mar 4, 2018 at 11:00 PM, Nikolay Izhikov <[hidden email]>
>> wrote:
>> > > Hello, Denis.
>> > >
>> > > > I would encourage you creating an IEP
>> > >
>> > > That is exactly what we want to do :)
>> > >
>> > > But seems I have not sufficient privileges to do it on Ignite wiki.
>> > >
>> > > https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
>> > >
>> > > Can you or someone give me such rights?
>> > >
>> > > В Чт, 01/03/2018 в 22:23 -0800, Denis Magda пишет:
>> > > > Dmitriy R., Nilokay,
>> > > >
>> > > > Thanks for the analysis and handout of the architectural design. No
>> doubts,
>> > > > it would be a valuable addition to Ignite.
>> > > >
>> > > > I would encourage you creating an IEP on the wiki and break the
>> work into
>> > > > pieces discussing specific part with the community.
>> > > >
>> > > > --
>> > > > Denis
>> > > >
>> > > >
>> > > > On Thu, Mar 1, 2018 at 9:29 PM, Nikolay Izhikov <
>> [hidden email]> wrote:
>> > > >
>> > > > > Hello, Dmitriy.
>> > > > >
>> > > > > Thank you for feedback!
>> > > > >
>> > > > > > Will it be supported?
>> > > > >
>> > > > > Yes.
>> > > > >
>> > > > > TDE shouldn't broke any of existing Ignite features.
>> > > > > It adds some encrypt/decrypt level when we writing and reading
>> pages
>> > > > > in/from PDS.
>> > > > >
>> > > > > В Пт, 02/03/2018 в 07:29 +0300, Dmitriy Setrakyan пишет:
>> > > > > > I have looked at the design, but could not find anything about
>> running
>> > > > >
>> > > > > SQL
>> > > > > > queries against the encrypted data. Will it be supported?
>> > > > > >
>> > > > > > D.
>> > > > > >
>> > > > > > On Thu, Mar 1, 2018 at 8:05 PM, Nikolay Izhikov <
>> [hidden email]>
>> > > > >
>> > > > > wrote:
>> > > > > >
>> > > > > > > Hell, Dima!
>> > > > > > >
>> > > > > > > Thank you for document!
>> > > > > > >
>> > > > > > > I'm ready to implement this feature with you.
>> > > > > > >
>> > > > > > > Igniters, please, share you thoughts about proposed design
>> > > > > > >
>> > > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>> > > > > > >
>> > > > > > > В Чт, 01/03/2018 в 15:46 +0300, Дмитрий Рябов пишет:
>> > > > > > > > Hello, Igniters!
>> > > > > > > >
>> > > > > > > > I investigated the issue and wrote some details in a draft
>> document
>> > > > > > > > [1]. I think we should made IEP for TDE because it is a big
>> change
>> > > > >
>> > > > > and
>> > > > > > > > should be described in a single place, but not in a message
>> > > > > > > > conversation.
>> > > > > > > > Please, look it and write your thoughts. What is not
>> understandable,
>> > > > > > > > what should be detailed or described?
>> > > > > > > >
>> > > > > > > > > Where are we going to store keys (MEK) physically? Would
>> it be
>> > > > >
>> > > > > PKCS#11
>> > > > > > > > > storage? Where we will store passwords to unlock storage
>> or it
>> > > > >
>> > > > > will be
>> > > > > > > > > responibilty of user?
>> > > > > > > >
>> > > > > > > > I think we should provide interface for MEK storage to let
>> users use
>> > > > > > > > storages they want. I suppose at the first step we should
>> provide
>> > > > >
>> > > > > very
>> > > > > > > > simple implementation, which will store MEK on every node
>> and MEK
>> > > > >
>> > > > > will
>> > > > > > > > be extracted by administrator during cluster activation
>> process. Once
>> > > > > > > > MEK is extracted from key store, we decrypt CEKs and
>> destroy open
>> > > > >
>> > > > > MEK,
>> > > > > > > > leaving open only cache keys.
>> > > > > > > >
>> > > > > > > > I think external storage is user's worry and we shouldn't
>> give users
>> > > > > > > > built-in external storage like Oracle Wallet or Microsoft
>> Azure Key
>> > > > > > > > Vault because it will increase Ignite's complexity too much.
>> > > > > > > >
>> > > > > > > > And yes, we should to comply with the standards like
>> PKCS#11.
>> > > > > > > >
>> > > > > > > > > One more thing is how "node gets MEK from coordinator",
>> if we send
>> > > > > > > > > cleartext MEK, such security becomes useless also.
>> > > > > > > >
>> > > > > > > > Yeah, that's why we should use secured connection. As I
>> know, we have
>> > > > > > > > SSL implementation over JDK implementation, am I right? But
>> we must
>> > > > > > > > ensure to use latest SSL/TLS version.
>> > > > > > > >
>> > > > > > > > [1] https://1drv.ms/w/s!AqZdfua4UpmuhneoVhOCiXSUBGIf
>> >
>> >
>>
>
>