Entity Key Store

[David Vediner]

Hi everyone -

Upon reviewing the entity engine encryption code it looks like the keys
for entity encryption are stored unencrypted in the database.  Because
of this, I can't really see the purpose of doing any encryption at all,
other than providing a minor layer of abstraction for a potential
hacker.  Obviously, if a hacker has gotten access to your database your
data is compromised, so securing your database is the most important.  
However, it seems to me that the point of encrypting data in the
database (since it is decrypted the instant it hits the entity engine)
is to protect that data from unauthorized sources, even if they have
access to your database (like your DBA, who really shouldn't be able to
read encrypted data).  I'm having a hard time seeing how the encryption
as it's implemented provides any security at all (say, of credit card
numbers) since if someone has gotten access to the encrypted data, they
can probably get access to the keys in the entity_key_store table (since
the data is only encrypted in the database, not in the application
layer).  In other words, if they have a need for the key that means they
have encrypted data, and if the data they have is encrypted, it seems
likely that they have somehow gotten access to the the database.

Of course the framework can only do so much - the people implementing
ofbiz really have a responsibility to secure their installation, and
anyone can take a very secure application and make it not so by their
configuration or implementation of it.  However, I think the system as
it is gives the *illusion* of security through encryption, which seems
even more dangerous than not providing encryption in the first place.

As I see it there are three options (I'm sure there are more) for
changing this in the code - 1) either move the key store to the
application layer, 2) re-encrypt the actual data so it would be
decrypted/encrypted once using the existing method and once using a new
key stored in the application layer (just double-encrypt everything), or
3) encrypt the keys in entity_key_store using a new key stored somewhere
in the application layer.

#1 has the "benefit" (perhaps from a cleanliness standpoint) of being
conceptually simple - you have data that is encrypted in the database
and you have a plaintext keys somewhere (now outside of the data layer)
to decrypt it.  No double encrypting.  #2 and #3 involve some sort of
double encryption; #2 just encrypts each piece of data twice using
different keys and #3 encrypts the encryption key itself.  I have
implemented #3 for our own code because it meant we only had to
re-encrypt the handful of encryption keys in the database, rather than
re-encrypt all the encrypted data in the database.  I didn't like #1
because I didn't want to think about where the best place to store the
keys are (and felt, apparently, that it was easier to figure out where
to stick a single encryption key).

So I changed the entity engine (when it needs to pull a key from the key
store) to decrypt that key first using a new plaintext key stored in the
entityengine.xml configuration file.  It uses AES (Rijndael) 256-bit
encryption using a simple AesCrypt utility class I created that's
basically a clone of the DesCrypt utility class, and the plaintext key
is run through a hash so a you can't just use the value in
entityengine.xml.  It's a really simple addition to the entity engine,
which was my goal, and an attacker can't compromise the encrypted data
by just gaining access to the database.  I'd be happy to dot the I's and
cross the T's in what little code I've written and submit it as a patch.

Also, perhaps someone who has actual experience with cryptanalysis
(particularly with the 3DES and Rijndael algorithms) can answer this
question: is there any practically useful extra information for
brute-forcing a 3DES encryption when you have a Rijndael-encrypted
version of the key?  That is, is it appreciably easier to crack some
3DES-encrypted data when you have a the key in encrypted form vs. when
you don't have any form of the key?  I doubt it, but really don't know.  
If it is easier, than the #3 method above is cryptographically inferior
to #2 and #1, although it might still be overall more secure than #1.

However, I'm open to the possibility that I haven't thought this through
completely enough and a) there is a better solution (likely) or b) I'm
just spinning my wheels here and the system as it is is actually secure
(although I know that it is not secure enough for our purposes, but I
admit that this might have to do with how we use our database).

Regards,
David Vediner

[David E Jones-2]

A couple of things to consider:

1. the initial decision to put the keys in the database was made because it avoids problems with keeping keys updated and consistent on multiple servers, and the database is typically more secure than the application servers (the database should never be accessible from the outside or external network, and so attacks often come through compromised web or application servers

2. it's tough to stop a resourceful DBA no matter where you put the keys; putting keys on the app server can keep them away from the DBAs if that is your primary concern, but you can also just put them in a separate database that the main DBA group does not have any access to ()this is fairly trivial with the delegator and such; anyone who has access to the app servers of course must necessarily have access to db access information plus the key locations, so it is impossible to avoid having someone in the organization with access to these things

3. for the database as well as the applications no employee should _ever_ access it with a generic account, they should always have and only use their own username and password and have explicit permissions setup as needed; in fact this is required by the Visa/MasterCard CC security standard group

-David



David Vediner wrote:

> Hi everyone -
>
> Upon reviewing the entity engine encryption code it looks like the keys
> for entity encryption are stored unencrypted in the database.  Because
> of this, I can't really see the purpose of doing any encryption at all,
> other than providing a minor layer of abstraction for a potential
> hacker.  Obviously, if a hacker has gotten access to your database your
> data is compromised, so securing your database is the most important.  
> However, it seems to me that the point of encrypting data in the
> database (since it is decrypted the instant it hits the entity engine)
> is to protect that data from unauthorized sources, even if they have
> access to your database (like your DBA, who really shouldn't be able to
> read encrypted data).  I'm having a hard time seeing how the encryption
> as it's implemented provides any security at all (say, of credit card
> numbers) since if someone has gotten access to the encrypted data, they
> can probably get access to the keys in the entity_key_store table (since
> the data is only encrypted in the database, not in the application
> layer).  In other words, if they have a need for the key that means they
> have encrypted data, and if the data they have is encrypted, it seems
> likely that they have somehow gotten access to the the database.
>
> Of course the framework can only do so much - the people implementing
> ofbiz really have a responsibility to secure their installation, and
> anyone can take a very secure application and make it not so by their
> configuration or implementation of it.  However, I think the system as
> it is gives the *illusion* of security through encryption, which seems
> even more dangerous than not providing encryption in the first place.
>
> As I see it there are three options (I'm sure there are more) for
> changing this in the code - 1) either move the key store to the
> application layer, 2) re-encrypt the actual data so it would be
> decrypted/encrypted once using the existing method and once using a new
> key stored in the application layer (just double-encrypt everything), or
> 3) encrypt the keys in entity_key_store using a new key stored somewhere
> in the application layer.
>
> #1 has the "benefit" (perhaps from a cleanliness standpoint) of being
> conceptually simple - you have data that is encrypted in the database
> and you have a plaintext keys somewhere (now outside of the data layer)
> to decrypt it.  No double encrypting.  #2 and #3 involve some sort of
> double encryption; #2 just encrypts each piece of data twice using
> different keys and #3 encrypts the encryption key itself.  I have
> implemented #3 for our own code because it meant we only had to
> re-encrypt the handful of encryption keys in the database, rather than
> re-encrypt all the encrypted data in the database.  I didn't like #1
> because I didn't want to think about where the best place to store the
> keys are (and felt, apparently, that it was easier to figure out where
> to stick a single encryption key).
>
> So I changed the entity engine (when it needs to pull a key from the key
> store) to decrypt that key first using a new plaintext key stored in the
> entityengine.xml configuration file.  It uses AES (Rijndael) 256-bit
> encryption using a simple AesCrypt utility class I created that's
> basically a clone of the DesCrypt utility class, and the plaintext key
> is run through a hash so a you can't just use the value in
> entityengine.xml.  It's a really simple addition to the entity engine,
> which was my goal, and an attacker can't compromise the encrypted data
> by just gaining access to the database.  I'd be happy to dot the I's and
> cross the T's in what little code I've written and submit it as a patch.
>
> Also, perhaps someone who has actual experience with cryptanalysis
> (particularly with the 3DES and Rijndael algorithms) can answer this
> question: is there any practically useful extra information for
> brute-forcing a 3DES encryption when you have a Rijndael-encrypted
> version of the key?  That is, is it appreciably easier to crack some
> 3DES-encrypted data when you have a the key in encrypted form vs. when
> you don't have any form of the key?  I doubt it, but really don't know.  
> If it is easier, than the #3 method above is cryptographically inferior
> to #2 and #1, although it might still be overall more secure than #1.
>
> However, I'm open to the possibility that I haven't thought this through
> completely enough and a) there is a better solution (likely) or b) I'm
> just spinning my wheels here and the system as it is is actually secure
> (although I know that it is not secure enough for our purposes, but I
> admit that this might have to do with how we use our database).
>
> Regards,
> David Vediner
>
>

[David Vediner]

Hi -

Good points.  I don't have to (not yet anyway) consider a server cluster
so I can be selfish and not worry about multiple machines.  And you're
right about the DB server being on an isolated network, of course, but
in reality most small installations can't/don't do that.  That may or
may not be an issue OFBiz should consider, though, I don't know.  And
yeah, you've got to trust the people you work with, too, because at some
point they'll probably need access to sensitive data.  And no matter how
hard you hide your encryption keys at least your application needs to
get a hold of them at some point or they're not very useful.  If not,
that's a pretty good indicator that you should have used a one-way hash,
or not stored the data in the first place.  :)

I guess my primary concern with the encryption as it is is that I don't
see what security it provides.  Worse than that, if it doesn't really
provide security it is at least providing the illusion of security to
those less diligent than they should be.  If someone gets into your
database, the encryption will not do anything to stop them, and yet the
encryption isn't a factor _unless_ you're in the database.. seems like a
catch-22.  So why encrypt anything?  Maybe I'm missing something
important here, or maybe the purpose of the encryption wasn't to provide
security in the first place and I'm reading a purpose into it that
wasn't ever intended?

Perhaps this isn't really a flaw in the entity engine design, however,
and it just needs to be emphasized in some future iteration of the setup
guide that unless you act otherwise the keys are stored in plaintext
alongside the encrypted data, and that it's a best practice to do
something like you suggest and use the entity engine to store them
somewhere else using a different delegator.  Or at least be aware that
if you don't do something about it, you should not rely on the
encryption to provide any layer of security.  Perhaps it's already in
there and I've just missed it.

I'm sure we're agreed that securing your database is much more important
than encrypting your stored data.  No one can crack the encryption if
they don't have the data in the first place, and it's not just the
encrypted data you're trying to protect, but generally all your data.

Good point about using a separate delegator...  I hadn't considered
that.  That might be a cleaner option and I'll look into it for our own
purposes (and using different delegators should solve another issue I'm
having with DB backups, actually), although unless I think that what
I've done with the keys is actually less secure I'll probably keep
encrypting the keys in the fashion I'm doing now.  But maybe that's how
it should be - each user should evaluate this key situation and see if
it works for their needs, and if not do something like I'm doing.

Thanks,
David Vediner



David E. Jones wrote:

>
> A couple of things to consider:
>
> 1. the initial decision to put the keys in the database was made
> because it avoids problems with keeping keys updated and consistent on
> multiple servers, and the database is typically more secure than the
> application servers (the database should never be accessible from the
> outside or external network, and so attacks often come through
> compromised web or application servers
>
> 2. it's tough to stop a resourceful DBA no matter where you put the
> keys; putting keys on the app server can keep them away from the DBAs
> if that is your primary concern, but you can also just put them in a
> separate database that the main DBA group does not have any access to
> ()this is fairly trivial with the delegator and such; anyone who has
> access to the app servers of course must necessarily have access to db
> access information plus the key locations, so it is impossible to
> avoid having someone in the organization with access to these things
>
> 3. for the database as well as the applications no employee should
> _ever_ access it with a generic account, they should always have and
> only use their own username and password and have explicit permissions
> setup as needed; in fact this is required by the Visa/MasterCard CC
> security standard group
>
> -David
>
>
>
> David Vediner wrote:
>> Hi everyone -
>>
>> Upon reviewing the entity engine encryption code it looks like the
>> keys for entity encryption are stored unencrypted in the database.  
>> Because of this, I can't really see the purpose of doing any
>> encryption at all, other than providing a minor layer of abstraction
>> for a potential hacker.  Obviously, if a hacker has gotten access to
>> your database your data is compromised, so securing your database is
>> the most important.  However, it seems to me that the point of
>> encrypting data in the database (since it is decrypted the instant it
>> hits the entity engine) is to protect that data from unauthorized
>> sources, even if they have access to your database (like your DBA,
>> who really shouldn't be able to read encrypted data).  I'm having a
>> hard time seeing how the encryption as it's implemented provides any
>> security at all (say, of credit card numbers) since if someone has
>> gotten access to the encrypted data, they can probably get access to
>> the keys in the entity_key_store table (since the data is only
>> encrypted in the database, not in the application layer).  In other
>> words, if they have a need for the key that means they have encrypted
>> data, and if the data they have is encrypted, it seems likely that
>> they have somehow gotten access to the the database.
>>
>> Of course the framework can only do so much - the people implementing
>> ofbiz really have a responsibility to secure their installation, and
>> anyone can take a very secure application and make it not so by their
>> configuration or implementation of it.  However, I think the system
>> as it is gives the *illusion* of security through encryption, which
>> seems even more dangerous than not providing encryption in the first
>> place.
>>
>> As I see it there are three options (I'm sure there are more) for
>> changing this in the code - 1) either move the key store to the
>> application layer, 2) re-encrypt the actual data so it would be
>> decrypted/encrypted once using the existing method and once using a
>> new key stored in the application layer (just double-encrypt
>> everything), or 3) encrypt the keys in entity_key_store using a new
>> key stored somewhere in the application layer.
>>
>> #1 has the "benefit" (perhaps from a cleanliness standpoint) of being
>> conceptually simple - you have data that is encrypted in the database
>> and you have a plaintext keys somewhere (now outside of the data
>> layer) to decrypt it.  No double encrypting.  #2 and #3 involve some
>> sort of double encryption; #2 just encrypts each piece of data twice
>> using different keys and #3 encrypts the encryption key itself.  I
>> have implemented #3 for our own code because it meant we only had to
>> re-encrypt the handful of encryption keys in the database, rather
>> than re-encrypt all the encrypted data in the database.  I didn't
>> like #1 because I didn't want to think about where the best place to
>> store the keys are (and felt, apparently, that it was easier to
>> figure out where to stick a single encryption key).
>>
>> So I changed the entity engine (when it needs to pull a key from the
>> key store) to decrypt that key first using a new plaintext key stored
>> in the entityengine.xml configuration file.  It uses AES (Rijndael)
>> 256-bit encryption using a simple AesCrypt utility class I created
>> that's basically a clone of the DesCrypt utility class, and the
>> plaintext key is run through a hash so a you can't just use the value
>> in entityengine.xml.  It's a really simple addition to the entity
>> engine, which was my goal, and an attacker can't compromise the
>> encrypted data by just gaining access to the database.  I'd be happy
>> to dot the I's and cross the T's in what little code I've written and
>> submit it as a patch.
>>
>> Also, perhaps someone who has actual experience with cryptanalysis
>> (particularly with the 3DES and Rijndael algorithms) can answer this
>> question: is there any practically useful extra information for
>> brute-forcing a 3DES encryption when you have a Rijndael-encrypted
>> version of the key?  That is, is it appreciably easier to crack some
>> 3DES-encrypted data when you have a the key in encrypted form vs.
>> when you don't have any form of the key?  I doubt it, but really
>> don't know.  If it is easier, than the #3 method above is
>> cryptographically inferior to #2 and #1, although it might still be
>> overall more secure than #1.
>>
>> However, I'm open to the possibility that I haven't thought this
>> through completely enough and a) there is a better solution (likely)
>> or b) I'm just spinning my wheels here and the system as it is is
>> actually secure (although I know that it is not secure enough for our
>> purposes, but I admit that this might have to do with how we use our
>> database).
>>
>> Regards,
>> David Vediner
>>
>>