Security Issues

[David E Jones-3]

Hello all.

I'm actually a little surprised we're still where we are on this, so  
I'm putting some time into this... understanding that it will annoy as  
many people as it pleases (at first anyway...).

In order to address various XSS and XSS-like security threats, I'd  
like to get some real and comprehensive stuff in place. Right now  
there are super-easy attacks that can be done, like putting JavaScript  
in a field during checkout that gets executed when a CSR (or anyone  
using the Order Manager) looks at the order, or someone looks at it in  
the Party Manager or wherever. That script can grab the session id and  
send it to a URL for session hijacking, or it can directly perform  
some action like marking the order as paid offline or creating a new  
admin account or changing the users password or whatever. The script  
could do anything the poor back-end user has access to do, and that's  
just an example.

The best issues on this are:

https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good  
review of OFBiz security and applicable comments, good tips to resolve)
https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original one,  
including my silly comment on it)

We have some simple code that does escaping for HTML chars, but it's  
not really used anywhere. Anyway, I think we need something more  
robust and comprehensive, especially given the fun ways of getting  
around filters and other things presented here:

http://ha.ckers.org/xss.html

What I'd like to do is add the OWASP ESAPI library, which is BSD  
licensed. There is a nice presentation about it as well here:

http://code.google.com/p/owasp-esapi-java/

and JavaDocs here:

http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html

======================================

So, there's a tool, now how/where to use it in OFBiz...

I think this will require a fair bit of work, and I know I'll miss  
things that are important in this first pass, but we can do some  
things to take care of the more obvious problems:

1. validate input: consider not allowing HTML in any field by default,  
and require an attribute set on service attributes or possibly even  
entity fields to say that restricted/safe HTML is allowed, or any HTML  
is allowed; this will break some things that actually need HTML  
initially, but fixing the broken things once found will be really easy

2. encode output: just in case HTML chars do get in somehow, we want  
to encode them so they are displayed literally and not interpreted as  
HTML by the browser; this will help avoid problems with messing up  
formatting when HTML is present, as well as helping with this security  
problem; this is easy to do in the various widgets (Screen, Form,  
Tree, Menu), and is tougher in FTL files if we want it encoded by  
default instead of manually using ?html on every field we want  
encoded, and I'd rather use the ESAPI encoder than the FTL one too;  
since much of this data is displayed right out of GenericValue  
objects, one possible solution is to change the GenericValue.get  
methods to do this encoding, and add a new get method that will not do  
encoding; this would handle the situations where the GenericValue is  
treated like a Map; this may also cause some crazy stuff to happen in  
places where gets are used in services and such and not in the UI...  
but I'm still thinking that through and am not sure if it will be a  
problem... it is kind of using bomb to swat a fly so collateral damage  
is likely

3. consider adding a token that is required for all requests in a  
session except the first one, use a constantly changing token, and  
have it added by the URL writing stuff based on a value in the  
session; this would change on every request, which is a pain because  
it means that any page in someone's browser other than the most  
recently rendered one would not work (a problem we have with the  
externalLoginKey stuff) unless we keep a certain number of the most  
recent tokens in the session and allow any of the last 10 or 20 or  
something to be used

4. related to #3, and relevant whether or not we do #3, add a unique  
token to all rendered forms and require that when processing the form  
input; if we only allow the tokens to be used once this also fits the  
common pattern used for eliminating accidental multiple submissions of  
the same form; this could be done easily with the Form Widget and the  
ServiceEventHandler (or perhaps all of the event handlers...), and  
more manually supported in other places like FTL forms; this would  
require some configuration, and again the annoying part is to cover as  
much as possible we would want this on by default which may cause  
problems for some things which would then need to changed to support  
it or disable it for that particular form and/or event

====================================

I'm really interested in hearing what others have to say about these.  
Personally I've avoided most of these types of things because they  
always tend to cause a dozen problems for every problem they solve.  
I've mentioned some concerns, but there are many more. Some issues may  
just make the application less usable because of restrictions on being  
able to do things like use the back button (IMO supporting that is a  
critical part of any web app that is worth anything) or having a bunch  
of false positives for security errors because of some funny scenario  
that was not anticipated (and this isn't an if thing, it's a when and  
how often thing).

-David

[Adrian Crum]

David E Jones wrote:
> 1. validate input: consider not allowing HTML in any field by default,
> and require an attribute set on service attributes or possibly even
> entity fields to say that restricted/safe HTML is allowed, or any HTML
> is allowed; this will break some things that actually need HTML
> initially, but fixing the broken things once found will be really easy

+1

> 2. encode output: just in case HTML chars do get in somehow, we want to
> encode them so they are displayed literally and not interpreted as HTML
> by the browser; this will help avoid problems with messing up formatting
> when HTML is present, as well as helping with this security problem;
> this is easy to do in the various widgets (Screen, Form, Tree, Menu),
> and is tougher in FTL files if we want it encoded by default instead of
> manually using ?html on every field we want encoded, and I'd rather use
> the ESAPI encoder than the FTL one too; since much of this data is
> displayed right out of GenericValue objects, one possible solution is to
> change the GenericValue.get methods to do this encoding, and add a new
> get method that will not do encoding; this would handle the situations
> where the GenericValue is treated like a Map; this may also cause some
> crazy stuff to happen in places where gets are used in services and such
> and not in the UI... but I'm still thinking that through and am not sure
> if it will be a problem... it is kind of using bomb to swat a fly so
> collateral damage is likely

What about having an FTL decorator class for GenericValue that does
escaping? That would be a simple modification to FreeMarkerWorker.java.


-Adrian

[Adrian Crum]

David E Jones wrote:
> The best issues on this are:
>
> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good review
> of OFBiz security and applicable comments, good tips to resolve)
> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original one,
> including my silly comment on it)

Before we get started, how about choosing ONE of the many issues to use
for collaboration, and close the rest? There seems to be four or five
issues that all describe the same problem, but in different areas of the
project.

-Adrian

[Jacques Le Roux]

+1, easy just have to refer to the main one to check, ie https://issues.apache.org/jira/browse/OFBIZ-1525

Jacques

From: "Adrian Crum" <[hidden email]>

> David E Jones wrote:
>> The best issues on this are:
>>
>> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good review
>> of OFBiz security and applicable comments, good tips to resolve)
>> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original one,
>> including my silly comment on it)
>
> Before we get started, how about choosing ONE of the many issues to use
> for collaboration, and close the rest? There seems to be four or five
> issues that all describe the same problem, but in different areas of the
> project.
>
> -Adrian
>

[euronymous]

Hi David

I'm Michele Orrù, the one who published on you jira the issue you mentioned:
https://issues.apache.org/jira/browse/OFBIZ-1959

Basically we are using Ofbiz trunk in our projects, when we need to build ecommerce, crm, facility management and so on: we also developed a few customization (for example to be able to download a bunch of digital products as a single zipped one (a work that required song track bundles), some customization and issue solving of JMS related funzionalities, and security).

I'm a penetration tester, more for passion than for work: I already helped other companies such as Konakart to secure their products ( take a look at my blog: http://antisnatchor.com/2008/12/22/konakart-2260-responsible-disclosure/ ). Security (intended as Web Application Security, not Role Based one) in Ofbiz is not so "high" as you mentioned in your complete post, and as I mentioned in my jira ticket.

Now, the right thing to do as I said, and as you accepted (I'm glad for this), is to integrate the filtering functionality of ESAPI inside Ofbiz. It will not be so easy, because Ofbiz structure is so huge and vast, but the points you cited in your post regarding XSS and XSRF protection are OK, from a security point of view.

I was thinking to proceed in two ways, when I will have a bit of time: first implement a servlet filter that we can declare in web.xml that validates input BEFORE an eventually malicious request can reach the application surface. In this way a user can choose if implement the filter in every ofbiz application, or just leave ofbiz without it. Second idea, research where to put the validation code of ESAPI and integrate it in Ofbiz. Maybe UtilHttp can extend ESAPI DefaultEncoder to protect from XSS: I need time to look at this solutions, and from a few days I'm researching it. Maybe we can exchange some ideas to finally secure Ofbiz from web security threats.

Output encoding is useful but can be implemented in a second moment, because if we filter in a good way the input, output should not be malicious. XSRF protection trough random token it will be maybe the most difficult part, and I think your proposal of store a number n of last tokens is not a good solution, because it only minimize XSRF (without actually eliminating it) restricting the attack window time.


Last thing: you're right when you speak about "I've avoided most of these types of things because they  
always tend to cause a dozen problems for every problem they solve." or "having a bunch  
of false positives for security errors because of some funny scenario  
that was not anticipated (and this isn't an if thing, it's a when and  
how often thing). ".....your points are correct, but the main point is that we (as Ofbiz users/developers) cannot leave Ofbiz in such an "insecure" state. Ofbiz is really an application that can change the market, and improving his security to maximum levels is definitely needed, and maybe an urgent TO-DO.

As I wrote here (I'm nickname: euronymous): http://sla.ckers.org/forum/read.php?3,25331,25334#msg-25334  there are a lot of production websites created with Ofbiz that are vulnerable to every attack I described in the jira issue: if we don't integrate security functions, almost no one is gonna put something like an application firewall in front of it in production.

Said this...I'm gonna research a bit about ESAPI integration.

Looking forward for your thoughts

Have a good weekend.

Michele

[Jacques Le Roux]

From: "David E Jones" <[hidden email]>

>
> Hello all.
>
> I'm actually a little surprised we're still where we are on this, so  I'm putting some time into this... understanding that it
> will annoy as  many people as it pleases (at first anyway...).
>
> In order to address various XSS and XSS-like security threats, I'd  like to get some real and comprehensive stuff in place. Right
> now  there are super-easy attacks that can be done, like putting JavaScript  in a field during checkout that gets executed when a
> CSR (or anyone  using the Order Manager) looks at the order, or someone looks at it in  the Party Manager or wherever. That script
> can grab the session id and  send it to a URL for session hijacking, or it can directly perform  some action like marking the
> order as paid offline or creating a new  admin account or changing the users password or whatever. The script  could do anything
> the poor back-end user has access to do, and that's  just an example.
>
> The best issues on this are:
>
> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good  review of OFBiz security and applicable comments, good tips to
> resolve)
> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original one,  including my silly comment on it)

:D

> We have some simple code that does escaping for HTML chars, but it's  not really used anywhere. Anyway, I think we need something
> more  robust and comprehensive, especially given the fun ways of getting  around filters and other things presented here:
>
> http://ha.ckers.org/xss.html
>
> What I'd like to do is add the OWASP ESAPI library, which is BSD  licensed. There is a nice presentation about it as well here:
>
> http://code.google.com/p/owasp-esapi-java/
>
> and JavaDocs here:
>
> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html

Wooww, with beautiful draws in (see Validator at least) !
Not sure how *ValidCreditCard (did not look into sourec) methods work but interesting...

And a very fast/good presentation (but in PowerPoint :/) here
http://owasp-esapi-java.googlecode.com/files/OWASP%20ESAPI.ppt

A big + 1 (just looked at presentation for now)

> ======================================
>
> So, there's a tool, now how/where to use it in OFBiz...
>
> I think this will require a fair bit of work, and I know I'll miss  things that are important in this first pass, but we can do
> some  things to take care of the more obvious problems:
>
> 1. validate input: consider not allowing HTML in any field by default,  and require an attribute set on service attributes or
> possibly even  entity fields to say that restricted/safe HTML is allowed, or any HTML  is allowed; this will break some things
> that actually need HTML  initially, but fixing the broken things once found will be really easy

+1

> 2. encode output: just in case HTML chars do get in somehow, we want  to encode them so they are displayed literally and not
> interpreted as  HTML by the browser; this will help avoid problems with messing up  formatting when HTML is present, as well as
> helping with this security  problem;

Will not that be a problem in Content Component ?

>this is easy to do in the various widgets (Screen, Form,  Tree, Menu), and is tougher in FTL files if we want it encoded by
>default instead of manually using ?html on every field we want  encoded, and I'd rather use the ESAPI encoder than the FTL one too;
>since much of this data is displayed right out of GenericValue  objects, one possible solution is to change the GenericValue.get
>methods to do this encoding, and add a new get method that will not do  encoding; this would handle the situations where the
>GenericValue is  treated like a Map; this may also cause some crazy stuff to happen in  places where gets are used in services and
>such and not in the UI...

Fritghening...

> but I'm still thinking that through and am not sure if it will be a  problem... it is kind of using bomb to swat a fly so
> collateral damage  is likely

:(

> 3. consider adding a token that is required for all requests in a  session except the first one, use a constantly changing token,
> and  have it added by the URL writing stuff based on a value in the  session; this would change on every request, which is a pain
> because  it means that any page in someone's browser other than the most  recently rendered one would not work (a problem we have
> with the  externalLoginKey stuff) unless we keep a certain number of the most  recent tokens in the session and allow any of the
> last 10 or 20 or  something to be used
>
> 4. related to #3, and relevant whether or not we do #3, add a unique  token to all rendered forms and require that when processing
> the form  input; if we only allow the tokens to be used once this also fits the  common pattern used for eliminating accidental
> multiple submissions of  the same form; this could be done easily with the Form Widget and the  ServiceEventHandler (or perhaps
> all of the event handlers...), and  more manually supported in other places like FTL forms; this would  require some
> configuration, and again the annoying part is to cover as  much as possible we would want this on by default which may cause
> problems for some things which would then need to changed to support  it or disable it for that particular form and/or event

It seems that's Michele (euronymous) saying <<it only minimizes XSRF (without actually eliminating it) restricting the attack window
time>> has a point there.
We may lean on his specific (hobby, best ones, with deep motivation ;o) knowledge and guide him where/if  he feels so ?

Jacques

> ====================================
>
> I'm really interested in hearing what others have to say about these.  Personally I've avoided most of these types of things
> because they  always tend to cause a dozen problems for every problem they solve.  I've mentioned some concerns, but there are
> many more. Some issues may  just make the application less usable because of restrictions on being  able to do things like use the
> back button (IMO supporting that is a  critical part of any web app that is worth anything) or having a bunch  of false positives
> for security errors because of some funny scenario  that was not anticipated (and this isn't an if thing, it's a when and  how
> often thing).
>
> -David
>
>

[euronymous]

jacques.le.roux wrote

It seems that's Michele (euronymous) saying <<it only minimizes XSRF (without actually eliminating it) restricting the attack window
time>> has a point there.
We may lean on his specific (hobby, best ones, with deep motivation ;o) knowledge and guide him where/if  he feels so ?

Jacques

Jacques, David, developers...

It would be a pleasure to work on Ofbiz security, from both a developer and attacker point of view...
It would be a good exercise for me, to deeply understand Ofbiz internals.

Maybe we can share your knowledge in ofbiz core, and my knowledge about web app security...
If you can point me in the right way, as Jacques said, I will develop the solution together with you...

Let me know

Michele

[pierre-29]

Hi all,

Here is a proposition on how to implement such XSS control:

 First we consider that all HHTP request should be filtering. So we
could add a filter into web.xml for each webapp that replaces a set of
dangerous characters by there HTML code.  By this way we can block all
XSS attacks for entire application.

 After filtering all requests, we should add a way to parameterise this.
So we could add 2 properties :
    - the first one to specifie a regex pattern that is used by filter
engine
    - the second one to disable filtering

 And to be very flexible we can set those properties (or attributes) on
3 levels :
     - request (from request-map)
     - webapp (for a complete webbapp)
     - application (main level)

And finaly we could consider that if there are no paramateres on request
level, then we look for webapp parameters. If there are no parameters on
webapp we look for application parameters.

By this way we could filter all request and set exeption or regex for a
particular request or webb-app or entire application.

What do you think about this.

Pierre

David E Jones wrote:

>
> Hello all.
>
> I'm actually a little surprised we're still where we are on this, so
> I'm putting some time into this... understanding that it will annoy as
> many people as it pleases (at first anyway...).
>
> In order to address various XSS and XSS-like security threats, I'd
> like to get some real and comprehensive stuff in place. Right now
> there are super-easy attacks that can be done, like putting JavaScript
> in a field during checkout that gets executed when a CSR (or anyone
> using the Order Manager) looks at the order, or someone looks at it in
> the Party Manager or wherever. That script can grab the session id and
> send it to a URL for session hijacking, or it can directly perform
> some action like marking the order as paid offline or creating a new
> admin account or changing the users password or whatever. The script
> could do anything the poor back-end user has access to do, and that's
> just an example.
>
> The best issues on this are:
>
> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good
> review of OFBiz security and applicable comments, good tips to resolve)
> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original one,
> including my silly comment on it)
>
> We have some simple code that does escaping for HTML chars, but it's
> not really used anywhere. Anyway, I think we need something more
> robust and comprehensive, especially given the fun ways of getting
> around filters and other things presented here:
>
> http://ha.ckers.org/xss.html
>
> What I'd like to do is add the OWASP ESAPI library, which is BSD
> licensed. There is a nice presentation about it as well here:
>
> http://code.google.com/p/owasp-esapi-java/
>
> and JavaDocs here:
>
> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>
> ======================================
>
> So, there's a tool, now how/where to use it in OFBiz...
>
> I think this will require a fair bit of work, and I know I'll miss
> things that are important in this first pass, but we can do some
> things to take care of the more obvious problems:
>
> 1. validate input: consider not allowing HTML in any field by default,
> and require an attribute set on service attributes or possibly even
> entity fields to say that restricted/safe HTML is allowed, or any HTML
> is allowed; this will break some things that actually need HTML
> initially, but fixing the broken things once found will be really easy
>
> 2. encode output: just in case HTML chars do get in somehow, we want
> to encode them so they are displayed literally and not interpreted as
> HTML by the browser; this will help avoid problems with messing up
> formatting when HTML is present, as well as helping with this security
> problem; this is easy to do in the various widgets (Screen, Form,
> Tree, Menu), and is tougher in FTL files if we want it encoded by
> default instead of manually using ?html on every field we want
> encoded, and I'd rather use the ESAPI encoder than the FTL one too;
> since much of this data is displayed right out of GenericValue
> objects, one possible solution is to change the GenericValue.get
> methods to do this encoding, and add a new get method that will not do
> encoding; this would handle the situations where the GenericValue is
> treated like a Map; this may also cause some crazy stuff to happen in
> places where gets are used in services and such and not in the UI...
> but I'm still thinking that through and am not sure if it will be a
> problem... it is kind of using bomb to swat a fly so collateral damage
> is likely
>
> 3. consider adding a token that is required for all requests in a
> session except the first one, use a constantly changing token, and
> have it added by the URL writing stuff based on a value in the
> session; this would change on every request, which is a pain because
> it means that any page in someone's browser other than the most
> recently rendered one would not work (a problem we have with the
> externalLoginKey stuff) unless we keep a certain number of the most
> recent tokens in the session and allow any of the last 10 or 20 or
> something to be used
>
> 4. related to #3, and relevant whether or not we do #3, add a unique
> token to all rendered forms and require that when processing the form
> input; if we only allow the tokens to be used once this also fits the
> common pattern used for eliminating accidental multiple submissions of
> the same form; this could be done easily with the Form Widget and the
> ServiceEventHandler (or perhaps all of the event handlers...), and
> more manually supported in other places like FTL forms; this would
> require some configuration, and again the annoying part is to cover as
> much as possible we would want this on by default which may cause
> problems for some things which would then need to changed to support
> it or disable it for that particular form and/or event
>
> ====================================
>
> I'm really interested in hearing what others have to say about these.
> Personally I've avoided most of these types of things because they
> always tend to cause a dozen problems for every problem they solve.
> I've mentioned some concerns, but there are many more. Some issues may
> just make the application less usable because of restrictions on being
> able to do things like use the back button (IMO supporting that is a
> critical part of any web app that is worth anything) or having a bunch
> of false positives for security errors because of some funny scenario
> that was not anticipated (and this isn't an if thing, it's a when and
> how often thing).
>
> -David
>
>
>

--
Pierre Gaudin,
Consultant Fonctionnel Neogia, Apache-OFBiz
ERP en logiciel Libre

Société Néréide
mobile : [phone]
bureau : [phone]
http://www.nereide.biz 

[Jacques Le Roux]

Hi Pierre,

From: "pierre" <[hidden email]>
> Hi all,
>
> Here is a proposition on how to implement such XSS control:
>
> First we consider that all HHTP request should be filtering. So we could add a filter into web.xml for each webapp that replaces a
> set of dangerous characters by there HTML code.  By this way we can block all XSS attacks for entire application.

Yes it makes sens indeed, that's what Michele also suggested in this thread, (with less details) :
http://www.nabble.com/Re%3A-Security-Issues-p21628377.html

> After filtering all requests, we should add a way to parameterise this. So we could add 2 properties :
>    - the first one to specifie a regex pattern that is used by filter engine
>    - the second one to disable filtering
>
> And to be very flexible we can set those properties (or attributes) on 3 levels :
>     - request (from request-map)
>     - webapp (for a complete webbapp)
>     - application (main level)

The more flexible the better.

> And finaly we could consider that if there are no paramateres on request level, then we look for webapp parameters. If there are
> no parameters on webapp we look for application parameters.
>
> By this way we could filter all request and set exeption or regex for a particular request or webb-app or entire application.
>
> What do you think about this.

Yes this will cover this security aspect, and sounds good to me.

Thanks

Jacques

> Pierre
>
> David E Jones wrote:
>>
>> Hello all.
>>
>> I'm actually a little surprised we're still where we are on this, so I'm putting some time into this... understanding that it
>> will annoy as many people as it pleases (at first anyway...).
>>
>> In order to address various XSS and XSS-like security threats, I'd like to get some real and comprehensive stuff in place. Right
>> now there are super-easy attacks that can be done, like putting JavaScript in a field during checkout that gets executed when a
>> CSR (or anyone using the Order Manager) looks at the order, or someone looks at it in the Party Manager or wherever. That script
>> can grab the session id and send it to a URL for session hijacking, or it can directly perform some action like marking the order
>> as paid offline or creating a new admin account or changing the users password or whatever. The script could do anything the poor
>> back-end user has access to do, and that's just an example.
>>
>> The best issues on this are:
>>
>> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good review of OFBiz security and applicable comments, good tips to
>> resolve)
>> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original one, including my silly comment on it)
>>
>> We have some simple code that does escaping for HTML chars, but it's not really used anywhere. Anyway, I think we need something
>> more robust and comprehensive, especially given the fun ways of getting around filters and other things presented here:
>>
>> http://ha.ckers.org/xss.html
>>
>> What I'd like to do is add the OWASP ESAPI library, which is BSD licensed. There is a nice presentation about it as well here:
>>
>> http://code.google.com/p/owasp-esapi-java/
>>
>> and JavaDocs here:
>>
>> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>>
>> ======================================
>>
>> So, there's a tool, now how/where to use it in OFBiz...
>>
>> I think this will require a fair bit of work, and I know I'll miss things that are important in this first pass, but we can do
>> some things to take care of the more obvious problems:
>>
>> 1. validate input: consider not allowing HTML in any field by default, and require an attribute set on service attributes or
>> possibly even entity fields to say that restricted/safe HTML is allowed, or any HTML is allowed; this will break some things that
>> actually need HTML initially, but fixing the broken things once found will be really easy
>>
>> 2. encode output: just in case HTML chars do get in somehow, we want to encode them so they are displayed literally and not
>> interpreted as HTML by the browser; this will help avoid problems with messing up formatting when HTML is present, as well as
>> helping with this security problem; this is easy to do in the various widgets (Screen, Form, Tree, Menu), and is tougher in FTL
>> files if we want it encoded by default instead of manually using ?html on every field we want encoded, and I'd rather use the
>> ESAPI encoder than the FTL one too; since much of this data is displayed right out of GenericValue objects, one possible solution
>> is to change the GenericValue.get methods to do this encoding, and add a new get method that will not do encoding; this would
>> handle the situations where the GenericValue is treated like a Map; this may also cause some crazy stuff to happen in places
>> where gets are used in services and such and not in the UI... but I'm still thinking that through and am not sure if it will be a
>> problem... it is kind of using bomb to swat a fly so collateral damage is likely
>>
>> 3. consider adding a token that is required for all requests in a session except the first one, use a constantly changing token,
>> and have it added by the URL writing stuff based on a value in the session; this would change on every request, which is a pain
>> because it means that any page in someone's browser other than the most recently rendered one would not work (a problem we have
>> with the externalLoginKey stuff) unless we keep a certain number of the most recent tokens in the session and allow any of the
>> last 10 or 20 or something to be used
>>
>> 4. related to #3, and relevant whether or not we do #3, add a unique token to all rendered forms and require that when processing
>> the form input; if we only allow the tokens to be used once this also fits the common pattern used for eliminating accidental
>> multiple submissions of the same form; this could be done easily with the Form Widget and the ServiceEventHandler (or perhaps all
>> of the event handlers...), and more manually supported in other places like FTL forms; this would require some configuration, and
>> again the annoying part is to cover as much as possible we would want this on by default which may cause problems for some things
>> which would then need to changed to support it or disable it for that particular form and/or event
>>
>> ====================================
>>
>> I'm really interested in hearing what others have to say about these. Personally I've avoided most of these types of things
>> because they always tend to cause a dozen problems for every problem they solve. I've mentioned some concerns, but there are many
>> more. Some issues may just make the application less usable because of restrictions on being able to do things like use the back
>> button (IMO supporting that is a critical part of any web app that is worth anything) or having a bunch of false positives for
>> security errors because of some funny scenario that was not anticipated (and this isn't an if thing, it's a when and how often
>> thing).
>>
>> -David
>>
>>
>>
>
>
> --
> Pierre Gaudin,
> Consultant Fonctionnel Neogia, Apache-OFBiz
> ERP en logiciel Libre
>
> Société Néréide
> mobile : [phone]
> bureau : [phone]
> http://www.nereide.biz
>

[pierre.gaudin]

Thank you Jacques,

Other opinions on this approach?

Can we work above?

Pierre

Jacques Le Roux wrote:

> Hi Pierre,
>
> From: "pierre" <[hidden email]>
>> Hi all,
>>
>> Here is a proposition on how to implement such XSS control:
>>
>> First we consider that all HHTP request should be filtering. So we
>> could add a filter into web.xml for each webapp that replaces a set
>> of dangerous characters by there HTML code.  By this way we can block
>> all XSS attacks for entire application.
>
> Yes it makes sens indeed, that's what Michele also suggested in this
> thread, (with less details) :
> http://www.nabble.com/Re%3A-Security-Issues-p21628377.html
>
>> After filtering all requests, we should add a way to parameterise
>> this. So we could add 2 properties :
>>    - the first one to specifie a regex pattern that is used by filter
>> engine
>>    - the second one to disable filtering
>>
>> And to be very flexible we can set those properties (or attributes)
>> on 3 levels :
>>     - request (from request-map)
>>     - webapp (for a complete webbapp)
>>     - application (main level)
>
> The more flexible the better.
>
>> And finaly we could consider that if there are no paramateres on
>> request level, then we look for webapp parameters. If there are no
>> parameters on webapp we look for application parameters.
>>
>> By this way we could filter all request and set exeption or regex for
>> a particular request or webb-app or entire application.
>>
>> What do you think about this.
>
> Yes this will cover this security aspect, and sounds good to me.
>
> Thanks
>
> Jacques
>
>> Pierre
>>
>> David E Jones wrote:
>>>
>>> Hello all.
>>>
>>> I'm actually a little surprised we're still where we are on this, so
>>> I'm putting some time into this... understanding that it will annoy
>>> as many people as it pleases (at first anyway...).
>>>
>>> In order to address various XSS and XSS-like security threats, I'd
>>> like to get some real and comprehensive stuff in place. Right now
>>> there are super-easy attacks that can be done, like putting
>>> JavaScript in a field during checkout that gets executed when a CSR
>>> (or anyone using the Order Manager) looks at the order, or someone
>>> looks at it in the Party Manager or wherever. That script can grab
>>> the session id and send it to a URL for session hijacking, or it can
>>> directly perform some action like marking the order as paid offline
>>> or creating a new admin account or changing the users password or
>>> whatever. The script could do anything the poor back-end user has
>>> access to do, and that's just an example.
>>>
>>> The best issues on this are:
>>>
>>> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good
>>> review of OFBiz security and applicable comments, good tips to resolve)
>>> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original
>>> one, including my silly comment on it)
>>>
>>> We have some simple code that does escaping for HTML chars, but it's
>>> not really used anywhere. Anyway, I think we need something more
>>> robust and comprehensive, especially given the fun ways of getting
>>> around filters and other things presented here:
>>>
>>> http://ha.ckers.org/xss.html
>>>
>>> What I'd like to do is add the OWASP ESAPI library, which is BSD
>>> licensed. There is a nice presentation about it as well here:
>>>
>>> http://code.google.com/p/owasp-esapi-java/
>>>
>>> and JavaDocs here:
>>>
>>> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>>>
>>> ======================================
>>>
>>> So, there's a tool, now how/where to use it in OFBiz...
>>>
>>> I think this will require a fair bit of work, and I know I'll miss
>>> things that are important in this first pass, but we can do some
>>> things to take care of the more obvious problems:
>>>
>>> 1. validate input: consider not allowing HTML in any field by
>>> default, and require an attribute set on service attributes or
>>> possibly even entity fields to say that restricted/safe HTML is
>>> allowed, or any HTML is allowed; this will break some things that
>>> actually need HTML initially, but fixing the broken things once
>>> found will be really easy
>>>
>>> 2. encode output: just in case HTML chars do get in somehow, we want
>>> to encode them so they are displayed literally and not interpreted
>>> as HTML by the browser; this will help avoid problems with messing
>>> up formatting when HTML is present, as well as helping with this
>>> security problem; this is easy to do in the various widgets (Screen,
>>> Form, Tree, Menu), and is tougher in FTL files if we want it encoded
>>> by default instead of manually using ?html on every field we want
>>> encoded, and I'd rather use the ESAPI encoder than the FTL one too;
>>> since much of this data is displayed right out of GenericValue
>>> objects, one possible solution is to change the GenericValue.get
>>> methods to do this encoding, and add a new get method that will not
>>> do encoding; this would handle the situations where the GenericValue
>>> is treated like a Map; this may also cause some crazy stuff to
>>> happen in places where gets are used in services and such and not in
>>> the UI... but I'm still thinking that through and am not sure if it
>>> will be a problem... it is kind of using bomb to swat a fly so
>>> collateral damage is likely
>>>
>>> 3. consider adding a token that is required for all requests in a
>>> session except the first one, use a constantly changing token, and
>>> have it added by the URL writing stuff based on a value in the
>>> session; this would change on every request, which is a pain because
>>> it means that any page in someone's browser other than the most
>>> recently rendered one would not work (a problem we have with the
>>> externalLoginKey stuff) unless we keep a certain number of the most
>>> recent tokens in the session and allow any of the last 10 or 20 or
>>> something to be used
>>>
>>> 4. related to #3, and relevant whether or not we do #3, add a unique
>>> token to all rendered forms and require that when processing the
>>> form input; if we only allow the tokens to be used once this also
>>> fits the common pattern used for eliminating accidental multiple
>>> submissions of the same form; this could be done easily with the
>>> Form Widget and the ServiceEventHandler (or perhaps all of the event
>>> handlers...), and more manually supported in other places like FTL
>>> forms; this would require some configuration, and again the annoying
>>> part is to cover as much as possible we would want this on by
>>> default which may cause problems for some things which would then
>>> need to changed to support it or disable it for that particular form
>>> and/or event
>>>
>>> ====================================
>>>
>>> I'm really interested in hearing what others have to say about
>>> these. Personally I've avoided most of these types of things because
>>> they always tend to cause a dozen problems for every problem they
>>> solve. I've mentioned some concerns, but there are many more. Some
>>> issues may just make the application less usable because of
>>> restrictions on being able to do things like use the back button
>>> (IMO supporting that is a critical part of any web app that is worth
>>> anything) or having a bunch of false positives for security errors
>>> because of some funny scenario that was not anticipated (and this
>>> isn't an if thing, it's a when and how often thing).
>>>
>>> -David
>>>
>>>
>>>
>>
>>
>> --
>> Pierre Gaudin,
>> Consultant Fonctionnel Neogia, Apache-OFBiz
>> ERP en logiciel Libre
>>
>> Société Néréide
>> mobile : [phone]
>> bureau : [phone]
>> http://www.nereide.biz
>>
>
>

--
Pierre Gaudin,
Consultant Fonctionnel Neogia, Apache-OFBiz
ERP en logiciel Libre

Société Néréide
mobile : [phone]
bureau : [phone]
http://www.nereide.biz 

[pierre-29]

Thank you Jacques,

Other opinions on this approach?

Can we work above?

Pierre

Jacques Le Roux wrote:

> Hi Pierre,
>
> From: "pierre" <[hidden email]>
>> Hi all,
>>
>> Here is a proposition on how to implement such XSS control:
>>
>> First we consider that all HHTP request should be filtering. So we
>> could add a filter into web.xml for each webapp that replaces a set
>> of dangerous characters by there HTML code.  By this way we can block
>> all XSS attacks for entire application.
>
> Yes it makes sens indeed, that's what Michele also suggested in this
> thread, (with less details) :
> http://www.nabble.com/Re%3A-Security-Issues-p21628377.html
>
>> After filtering all requests, we should add a way to parameterise
>> this. So we could add 2 properties :
>>    - the first one to specifie a regex pattern that is used by filter
>> engine
>>    - the second one to disable filtering
>>
>> And to be very flexible we can set those properties (or attributes)
>> on 3 levels :
>>     - request (from request-map)
>>     - webapp (for a complete webbapp)
>>     - application (main level)
>
> The more flexible the better.
>
>> And finaly we could consider that if there are no paramateres on
>> request level, then we look for webapp parameters. If there are no
>> parameters on webapp we look for application parameters.
>>
>> By this way we could filter all request and set exeption or regex for
>> a particular request or webb-app or entire application.
>>
>> What do you think about this.
>
> Yes this will cover this security aspect, and sounds good to me.
>
> Thanks
>
> Jacques
>
>> Pierre
>>
>> David E Jones wrote:
>>>
>>> Hello all.
>>>
>>> I'm actually a little surprised we're still where we are on this, so
>>> I'm putting some time into this... understanding that it will annoy
>>> as many people as it pleases (at first anyway...).
>>>
>>> In order to address various XSS and XSS-like security threats, I'd
>>> like to get some real and comprehensive stuff in place. Right now
>>> there are super-easy attacks that can be done, like putting
>>> JavaScript in a field during checkout that gets executed when a CSR
>>> (or anyone using the Order Manager) looks at the order, or someone
>>> looks at it in the Party Manager or wherever. That script can grab
>>> the session id and send it to a URL for session hijacking, or it can
>>> directly perform some action like marking the order as paid offline
>>> or creating a new admin account or changing the users password or
>>> whatever. The script could do anything the poor back-end user has
>>> access to do, and that's just an example.
>>>
>>> The best issues on this are:
>>>
>>> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good
>>> review of OFBiz security and applicable comments, good tips to resolve)
>>> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original
>>> one, including my silly comment on it)
>>>
>>> We have some simple code that does escaping for HTML chars, but it's
>>> not really used anywhere. Anyway, I think we need something more
>>> robust and comprehensive, especially given the fun ways of getting
>>> around filters and other things presented here:
>>>
>>> http://ha.ckers.org/xss.html
>>>
>>> What I'd like to do is add the OWASP ESAPI library, which is BSD
>>> licensed. There is a nice presentation about it as well here:
>>>
>>> http://code.google.com/p/owasp-esapi-java/
>>>
>>> and JavaDocs here:
>>>
>>> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>>>
>>> ======================================
>>>
>>> So, there's a tool, now how/where to use it in OFBiz...
>>>
>>> I think this will require a fair bit of work, and I know I'll miss
>>> things that are important in this first pass, but we can do some
>>> things to take care of the more obvious problems:
>>>
>>> 1. validate input: consider not allowing HTML in any field by
>>> default, and require an attribute set on service attributes or
>>> possibly even entity fields to say that restricted/safe HTML is
>>> allowed, or any HTML is allowed; this will break some things that
>>> actually need HTML initially, but fixing the broken things once
>>> found will be really easy
>>>
>>> 2. encode output: just in case HTML chars do get in somehow, we want
>>> to encode them so they are displayed literally and not interpreted
>>> as HTML by the browser; this will help avoid problems with messing
>>> up formatting when HTML is present, as well as helping with this
>>> security problem; this is easy to do in the various widgets (Screen,
>>> Form, Tree, Menu), and is tougher in FTL files if we want it encoded
>>> by default instead of manually using ?html on every field we want
>>> encoded, and I'd rather use the ESAPI encoder than the FTL one too;
>>> since much of this data is displayed right out of GenericValue
>>> objects, one possible solution is to change the GenericValue.get
>>> methods to do this encoding, and add a new get method that will not
>>> do encoding; this would handle the situations where the GenericValue
>>> is treated like a Map; this may also cause some crazy stuff to
>>> happen in places where gets are used in services and such and not in
>>> the UI... but I'm still thinking that through and am not sure if it
>>> will be a problem... it is kind of using bomb to swat a fly so
>>> collateral damage is likely
>>>
>>> 3. consider adding a token that is required for all requests in a
>>> session except the first one, use a constantly changing token, and
>>> have it added by the URL writing stuff based on a value in the
>>> session; this would change on every request, which is a pain because
>>> it means that any page in someone's browser other than the most
>>> recently rendered one would not work (a problem we have with the
>>> externalLoginKey stuff) unless we keep a certain number of the most
>>> recent tokens in the session and allow any of the last 10 or 20 or
>>> something to be used
>>>
>>> 4. related to #3, and relevant whether or not we do #3, add a unique
>>> token to all rendered forms and require that when processing the
>>> form input; if we only allow the tokens to be used once this also
>>> fits the common pattern used for eliminating accidental multiple
>>> submissions of the same form; this could be done easily with the
>>> Form Widget and the ServiceEventHandler (or perhaps all of the event
>>> handlers...), and more manually supported in other places like FTL
>>> forms; this would require some configuration, and again the annoying
>>> part is to cover as much as possible we would want this on by
>>> default which may cause problems for some things which would then
>>> need to changed to support it or disable it for that particular form
>>> and/or event
>>>
>>> ====================================
>>>
>>> I'm really interested in hearing what others have to say about
>>> these. Personally I've avoided most of these types of things because
>>> they always tend to cause a dozen problems for every problem they
>>> solve. I've mentioned some concerns, but there are many more. Some
>>> issues may just make the application less usable because of
>>> restrictions on being able to do things like use the back button
>>> (IMO supporting that is a critical part of any web app that is worth
>>> anything) or having a bunch of false positives for security errors
>>> because of some funny scenario that was not anticipated (and this
>>> isn't an if thing, it's a when and how often thing).
>>>
>>> -David
>>>
>>>
>>>
>>
>>
>> --
>> Pierre Gaudin,
>> Consultant Fonctionnel Neogia, Apache-OFBiz
>> ERP en logiciel Libre
>>
>> Société Néréide
>> mobile : [phone]
>> bureau : [phone]
>> http://www.nereide.biz
>>
>
>

--
Pierre Gaudin,
Consultant Fonctionnel Neogia, Apache-OFBiz
ERP en logiciel Libre

Société Néréide
mobile : [phone]
bureau : [phone]
http://www.nereide.biz 

[Jacques Le Roux]

https://issues.apache.org/jira/browse/OFBIZ-1525
I think any work on these tasks would be greatly appreciated. BTW I don't think any other Jira issue is needed ;o)

If anybody is already working on one of those tasks please chime in ...

Thanks

Jacques

From: "pierre" <[hidden email]>

> Thank you Jacques,
>
> Other opinions on this approach?
>
> Can we work above?
>
> Pierre
>
> Jacques Le Roux wrote:
>> Hi Pierre,
>>
>> From: "pierre" <[hidden email]>
>>> Hi all,
>>>
>>> Here is a proposition on how to implement such XSS control:
>>>
>>> First we consider that all HHTP request should be filtering. So we could add a filter into web.xml for each webapp that replaces
>>> a set of dangerous characters by there HTML code.  By this way we can block all XSS attacks for entire application.
>>
>> Yes it makes sens indeed, that's what Michele also suggested in this thread, (with less details) :
>> http://www.nabble.com/Re%3A-Security-Issues-p21628377.html
>>
>>> After filtering all requests, we should add a way to parameterise this. So we could add 2 properties :
>>>    - the first one to specifie a regex pattern that is used by filter engine
>>>    - the second one to disable filtering
>>>
>>> And to be very flexible we can set those properties (or attributes) on 3 levels :
>>>     - request (from request-map)
>>>     - webapp (for a complete webbapp)
>>>     - application (main level)
>>
>> The more flexible the better.
>>
>>> And finaly we could consider that if there are no paramateres on request level, then we look for webapp parameters. If there are
>>> no parameters on webapp we look for application parameters.
>>>
>>> By this way we could filter all request and set exeption or regex for a particular request or webb-app or entire application.
>>>
>>> What do you think about this.
>>
>> Yes this will cover this security aspect, and sounds good to me.
>>
>> Thanks
>>
>> Jacques
>>
>>> Pierre
>>>
>>> David E Jones wrote:
>>>>
>>>> Hello all.
>>>>
>>>> I'm actually a little surprised we're still where we are on this, so I'm putting some time into this... understanding that it
>>>> will annoy as many people as it pleases (at first anyway...).
>>>>
>>>> In order to address various XSS and XSS-like security threats, I'd like to get some real and comprehensive stuff in place.
>>>> Right now there are super-easy attacks that can be done, like putting JavaScript in a field during checkout that gets executed
>>>> when a CSR (or anyone using the Order Manager) looks at the order, or someone looks at it in the Party Manager or wherever.
>>>> That script can grab the session id and send it to a URL for session hijacking, or it can directly perform some action like
>>>> marking the order as paid offline or creating a new admin account or changing the users password or whatever. The script could
>>>> do anything the poor back-end user has access to do, and that's just an example.
>>>>
>>>> The best issues on this are:
>>>>
>>>> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good review of OFBiz security and applicable comments, good tips
>>>> to resolve)
>>>> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original one, including my silly comment on it)
>>>>
>>>> We have some simple code that does escaping for HTML chars, but it's not really used anywhere. Anyway, I think we need
>>>> something more robust and comprehensive, especially given the fun ways of getting around filters and other things presented
>>>> here:
>>>>
>>>> http://ha.ckers.org/xss.html
>>>>
>>>> What I'd like to do is add the OWASP ESAPI library, which is BSD licensed. There is a nice presentation about it as well here:
>>>>
>>>> http://code.google.com/p/owasp-esapi-java/
>>>>
>>>> and JavaDocs here:
>>>>
>>>> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>>>>
>>>> ======================================
>>>>
>>>> So, there's a tool, now how/where to use it in OFBiz...
>>>>
>>>> I think this will require a fair bit of work, and I know I'll miss things that are important in this first pass, but we can do
>>>> some things to take care of the more obvious problems:
>>>>
>>>> 1. validate input: consider not allowing HTML in any field by default, and require an attribute set on service attributes or
>>>> possibly even entity fields to say that restricted/safe HTML is allowed, or any HTML is allowed; this will break some things
>>>> that actually need HTML initially, but fixing the broken things once found will be really easy
>>>>
>>>> 2. encode output: just in case HTML chars do get in somehow, we want to encode them so they are displayed literally and not
>>>> interpreted as HTML by the browser; this will help avoid problems with messing up formatting when HTML is present, as well as
>>>> helping with this security problem; this is easy to do in the various widgets (Screen, Form, Tree, Menu), and is tougher in FTL
>>>> files if we want it encoded by default instead of manually using ?html on every field we want encoded, and I'd rather use the
>>>> ESAPI encoder than the FTL one too; since much of this data is displayed right out of GenericValue objects, one possible
>>>> solution is to change the GenericValue.get methods to do this encoding, and add a new get method that will not do encoding;
>>>> this would handle the situations where the GenericValue is treated like a Map; this may also cause some crazy stuff to happen
>>>> in places where gets are used in services and such and not in the UI... but I'm still thinking that through and am not sure if
>>>> it will be a problem... it is kind of using bomb to swat a fly so collateral damage is likely
>>>>
>>>> 3. consider adding a token that is required for all requests in a session except the first one, use a constantly changing
>>>> token, and have it added by the URL writing stuff based on a value in the session; this would change on every request, which is
>>>> a pain because it means that any page in someone's browser other than the most recently rendered one would not work (a problem
>>>> we have with the externalLoginKey stuff) unless we keep a certain number of the most recent tokens in the session and allow any
>>>> of the last 10 or 20 or something to be used
>>>>
>>>> 4. related to #3, and relevant whether or not we do #3, add a unique token to all rendered forms and require that when
>>>> processing the form input; if we only allow the tokens to be used once this also fits the common pattern used for eliminating
>>>> accidental multiple submissions of the same form; this could be done easily with the Form Widget and the ServiceEventHandler
>>>> (or perhaps all of the event handlers...), and more manually supported in other places like FTL forms; this would require some
>>>> configuration, and again the annoying part is to cover as much as possible we would want this on by default which may cause
>>>> problems for some things which would then need to changed to support it or disable it for that particular form and/or event
>>>>
>>>> ====================================
>>>>
>>>> I'm really interested in hearing what others have to say about these. Personally I've avoided most of these types of things
>>>> because they always tend to cause a dozen problems for every problem they solve. I've mentioned some concerns, but there are
>>>> many more. Some issues may just make the application less usable because of restrictions on being able to do things like use
>>>> the back button (IMO supporting that is a critical part of any web app that is worth anything) or having a bunch of false
>>>> positives for security errors because of some funny scenario that was not anticipated (and this isn't an if thing, it's a when
>>>> and how often thing).
>>>>
>>>> -David
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Pierre Gaudin,
>>> Consultant Fonctionnel Neogia, Apache-OFBiz
>>> ERP en logiciel Libre
>>>
>>> Société Néréide
>>> mobile : [phone]
>>> bureau : [phone]
>>> http://www.nereide.biz
>>>
>>
>>
>
>
> --
> Pierre Gaudin,
> Consultant Fonctionnel Neogia, Apache-OFBiz
> ERP en logiciel Libre
>
> Société Néréide
> mobile : [phone]
> bureau : [phone]
> http://www.nereide.biz
>

[Philipp Hoppen]

Hi Pierre

I took a look at the ESAPI library David suggested and tried to use it
in OFBiz for input filtering. It basically works as you described by a
filter specified in web.xml. There is a "SafeHTTPFilter" class that uses
a "SafeRequest" and a "SafeResponse" which call a "Validator" object
that is parametrized using regular expressions defined in
ESAPI.properties. The ESAPI reference implementations and default
property values are very restrictive (for example, leaving a form
parameter empty is not possible), so these have to be customized for use
in OFBiz. The good thing about a servlet filter is that it is used very
early in the processing and doesn't require to modify code, but I don't
know yet how we can make the link to an entity or service field to allow
HTML for specific parameters...

A Validator can then probably be used for output encoding too (maybe in
GenericValue.get as David suggested).

As we're interested in fixing these security issues in our OFBiz
projects (i made a Jira issue some weeks ago,
https://issues.apache.org/jira/browse/OFBIZ-2121, but it didn't get any
attention), I can work at least 1 day per week on this stuff. It would
be nice if we could work on this together in the same direction...

--
Philipp Hoppen,
nowhow solutions AG, Laupenstrasse 1, CH-3008 Bern
Phone [phone] http://www.nowhow.ch


pierre schrieb:

> Thank you Jacques,
>
> Other opinions on this approach?
>
> Can we work above?
>
> Pierre
>
> Jacques Le Roux wrote:
>> Hi Pierre,
>>
>> From: "pierre" <[hidden email]>
>>> Hi all,
>>>
>>> Here is a proposition on how to implement such XSS control:
>>>
>>> First we consider that all HHTP request should be filtering. So we
>>> could add a filter into web.xml for each webapp that replaces a set
>>> of dangerous characters by there HTML code.  By this way we can
>>> block all XSS attacks for entire application.
>>
>> Yes it makes sens indeed, that's what Michele also suggested in this
>> thread, (with less details) :
>> http://www.nabble.com/Re%3A-Security-Issues-p21628377.html
>>
>>> After filtering all requests, we should add a way to parameterise
>>> this. So we could add 2 properties :
>>>    - the first one to specifie a regex pattern that is used by
>>> filter engine
>>>    - the second one to disable filtering
>>>
>>> And to be very flexible we can set those properties (or attributes)
>>> on 3 levels :
>>>     - request (from request-map)
>>>     - webapp (for a complete webbapp)
>>>     - application (main level)
>>
>> The more flexible the better.
>>
>>> And finaly we could consider that if there are no paramateres on
>>> request level, then we look for webapp parameters. If there are no
>>> parameters on webapp we look for application parameters.
>>>
>>> By this way we could filter all request and set exeption or regex
>>> for a particular request or webb-app or entire application.
>>>
>>> What do you think about this.
>>
>> Yes this will cover this security aspect, and sounds good to me.
>>
>> Thanks
>>
>> Jacques
>>
>>> Pierre
>>>
>>> David E Jones wrote:
>>>>
>>>> Hello all.
>>>>
>>>> I'm actually a little surprised we're still where we are on this,
>>>> so I'm putting some time into this... understanding that it will
>>>> annoy as many people as it pleases (at first anyway...).
>>>>
>>>> In order to address various XSS and XSS-like security threats, I'd
>>>> like to get some real and comprehensive stuff in place. Right now
>>>> there are super-easy attacks that can be done, like putting
>>>> JavaScript in a field during checkout that gets executed when a CSR
>>>> (or anyone using the Order Manager) looks at the order, or someone
>>>> looks at it in the Party Manager or wherever. That script can grab
>>>> the session id and send it to a URL for session hijacking, or it
>>>> can directly perform some action like marking the order as paid
>>>> offline or creating a new admin account or changing the users
>>>> password or whatever. The script could do anything the poor
>>>> back-end user has access to do, and that's just an example.
>>>>
>>>> The best issues on this are:
>>>>
>>>> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good
>>>> review of OFBiz security and applicable comments, good tips to
>>>> resolve)
>>>> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original
>>>> one, including my silly comment on it)
>>>>
>>>> We have some simple code that does escaping for HTML chars, but
>>>> it's not really used anywhere. Anyway, I think we need something
>>>> more robust and comprehensive, especially given the fun ways of
>>>> getting around filters and other things presented here:
>>>>
>>>> http://ha.ckers.org/xss.html
>>>>
>>>> What I'd like to do is add the OWASP ESAPI library, which is BSD
>>>> licensed. There is a nice presentation about it as well here:
>>>>
>>>> http://code.google.com/p/owasp-esapi-java/
>>>>
>>>> and JavaDocs here:
>>>>
>>>> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>>>>
>>>> ======================================
>>>>
>>>> So, there's a tool, now how/where to use it in OFBiz...
>>>>
>>>> I think this will require a fair bit of work, and I know I'll miss
>>>> things that are important in this first pass, but we can do some
>>>> things to take care of the more obvious problems:
>>>>
>>>> 1. validate input: consider not allowing HTML in any field by
>>>> default, and require an attribute set on service attributes or
>>>> possibly even entity fields to say that restricted/safe HTML is
>>>> allowed, or any HTML is allowed; this will break some things that
>>>> actually need HTML initially, but fixing the broken things once
>>>> found will be really easy
>>>>
>>>> 2. encode output: just in case HTML chars do get in somehow, we
>>>> want to encode them so they are displayed literally and not
>>>> interpreted as HTML by the browser; this will help avoid problems
>>>> with messing up formatting when HTML is present, as well as helping
>>>> with this security problem; this is easy to do in the various
>>>> widgets (Screen, Form, Tree, Menu), and is tougher in FTL files if
>>>> we want it encoded by default instead of manually using ?html on
>>>> every field we want encoded, and I'd rather use the ESAPI encoder
>>>> than the FTL one too; since much of this data is displayed right
>>>> out of GenericValue objects, one possible solution is to change the
>>>> GenericValue.get methods to do this encoding, and add a new get
>>>> method that will not do encoding; this would handle the situations
>>>> where the GenericValue is treated like a Map; this may also cause
>>>> some crazy stuff to happen in places where gets are used in
>>>> services and such and not in the UI... but I'm still thinking that
>>>> through and am not sure if it will be a problem... it is kind of
>>>> using bomb to swat a fly so collateral damage is likely
>>>>
>>>> 3. consider adding a token that is required for all requests in a
>>>> session except the first one, use a constantly changing token, and
>>>> have it added by the URL writing stuff based on a value in the
>>>> session; this would change on every request, which is a pain
>>>> because it means that any page in someone's browser other than the
>>>> most recently rendered one would not work (a problem we have with
>>>> the externalLoginKey stuff) unless we keep a certain number of the
>>>> most recent tokens in the session and allow any of the last 10 or
>>>> 20 or something to be used
>>>>
>>>> 4. related to #3, and relevant whether or not we do #3, add a
>>>> unique token to all rendered forms and require that when processing
>>>> the form input; if we only allow the tokens to be used once this
>>>> also fits the common pattern used for eliminating accidental
>>>> multiple submissions of the same form; this could be done easily
>>>> with the Form Widget and the ServiceEventHandler (or perhaps all of
>>>> the event handlers...), and more manually supported in other places
>>>> like FTL forms; this would require some configuration, and again
>>>> the annoying part is to cover as much as possible we would want
>>>> this on by default which may cause problems for some things which
>>>> would then need to changed to support it or disable it for that
>>>> particular form and/or event
>>>>
>>>> ====================================
>>>>
>>>> I'm really interested in hearing what others have to say about
>>>> these. Personally I've avoided most of these types of things
>>>> because they always tend to cause a dozen problems for every
>>>> problem they solve. I've mentioned some concerns, but there are
>>>> many more. Some issues may just make the application less usable
>>>> because of restrictions on being able to do things like use the
>>>> back button (IMO supporting that is a critical part of any web app
>>>> that is worth anything) or having a bunch of false positives for
>>>> security errors because of some funny scenario that was not
>>>> anticipated (and this isn't an if thing, it's a when and how often
>>>> thing).
>>>>
>>>> -David
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Pierre Gaudin,
>>> Consultant Fonctionnel Neogia, Apache-OFBiz
>>> ERP en logiciel Libre
>>>
>>> Société Néréide
>>> mobile : [phone]
>>> bureau : [phone]
>>> http://www.nereide.biz
>>>
>>
>>
>
>

[Jacques Le Roux]

I took care of it, now it's related to https://issues.apache.org/jira/browse/OFBIZ-1525
Thanks for your help on this...

Jacques

From: "Philipp Hoppen" <[hidden email]>

> Hi Pierre
>
> I took a look at the ESAPI library David suggested and tried to use it in OFBiz for input filtering. It basically works as you
> described by a filter specified in web.xml. There is a "SafeHTTPFilter" class that uses a "SafeRequest" and a "SafeResponse" which
> call a "Validator" object that is parametrized using regular expressions defined in ESAPI.properties. The ESAPI reference
> implementations and default property values are very restrictive (for example, leaving a form parameter empty is not possible), so
> these have to be customized for use in OFBiz. The good thing about a servlet filter is that it is used very early in the
> processing and doesn't require to modify code, but I don't know yet how we can make the link to an entity or service field to
> allow HTML for specific parameters...
>
> A Validator can then probably be used for output encoding too (maybe in GenericValue.get as David suggested).
>
> As we're interested in fixing these security issues in our OFBiz projects (i made a Jira issue some weeks ago,
> https://issues.apache.org/jira/browse/OFBIZ-2121, but it didn't get any attention), I can work at least 1 day per week on this
> stuff. It would be nice if we could work on this together in the same direction...
>
> --
> Philipp Hoppen,
> nowhow solutions AG, Laupenstrasse 1, CH-3008 Bern
> Phone [phone] http://www.nowhow.ch
>
>
> pierre schrieb:
>> Thank you Jacques,
>>
>> Other opinions on this approach?
>>
>> Can we work above?
>>
>> Pierre
>>
>> Jacques Le Roux wrote:
>>> Hi Pierre,
>>>
>>> From: "pierre" <[hidden email]>
>>>> Hi all,
>>>>
>>>> Here is a proposition on how to implement such XSS control:
>>>>
>>>> First we consider that all HHTP request should be filtering. So we could add a filter into web.xml for each webapp that
>>>> replaces a set of dangerous characters by there HTML code.  By this way we can block all XSS attacks for entire application.
>>>
>>> Yes it makes sens indeed, that's what Michele also suggested in this thread, (with less details) :
>>> http://www.nabble.com/Re%3A-Security-Issues-p21628377.html
>>>
>>>> After filtering all requests, we should add a way to parameterise this. So we could add 2 properties :
>>>>    - the first one to specifie a regex pattern that is used by filter engine
>>>>    - the second one to disable filtering
>>>>
>>>> And to be very flexible we can set those properties (or attributes) on 3 levels :
>>>>     - request (from request-map)
>>>>     - webapp (for a complete webbapp)
>>>>     - application (main level)
>>>
>>> The more flexible the better.
>>>
>>>> And finaly we could consider that if there are no paramateres on request level, then we look for webapp parameters. If there
>>>> are no parameters on webapp we look for application parameters.
>>>>
>>>> By this way we could filter all request and set exeption or regex for a particular request or webb-app or entire application.
>>>>
>>>> What do you think about this.
>>>
>>> Yes this will cover this security aspect, and sounds good to me.
>>>
>>> Thanks
>>>
>>> Jacques
>>>
>>>> Pierre
>>>>
>>>> David E Jones wrote:
>>>>>
>>>>> Hello all.
>>>>>
>>>>> I'm actually a little surprised we're still where we are on this, so I'm putting some time into this... understanding that it
>>>>> will annoy as many people as it pleases (at first anyway...).
>>>>>
>>>>> In order to address various XSS and XSS-like security threats, I'd like to get some real and comprehensive stuff in place.
>>>>> Right now there are super-easy attacks that can be done, like putting JavaScript in a field during checkout that gets executed
>>>>> when a CSR (or anyone using the Order Manager) looks at the order, or someone looks at it in the Party Manager or wherever.
>>>>> That script can grab the session id and send it to a URL for session hijacking, or it can directly perform some action like
>>>>> marking the order as paid offline or creating a new admin account or changing the users password or whatever. The script could
>>>>> do anything the poor back-end user has access to do, and that's just an example.
>>>>>
>>>>> The best issues on this are:
>>>>>
>>>>> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good review of OFBiz security and applicable comments, good tips
>>>>> to resolve)
>>>>> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original one, including my silly comment on it)
>>>>>
>>>>> We have some simple code that does escaping for HTML chars, but it's not really used anywhere. Anyway, I think we need
>>>>> something more robust and comprehensive, especially given the fun ways of getting around filters and other things presented
>>>>> here:
>>>>>
>>>>> http://ha.ckers.org/xss.html
>>>>>
>>>>> What I'd like to do is add the OWASP ESAPI library, which is BSD licensed. There is a nice presentation about it as well here:
>>>>>
>>>>> http://code.google.com/p/owasp-esapi-java/
>>>>>
>>>>> and JavaDocs here:
>>>>>
>>>>> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>>>>>
>>>>> ======================================
>>>>>
>>>>> So, there's a tool, now how/where to use it in OFBiz...
>>>>>
>>>>> I think this will require a fair bit of work, and I know I'll miss things that are important in this first pass, but we can do
>>>>> some things to take care of the more obvious problems:
>>>>>
>>>>> 1. validate input: consider not allowing HTML in any field by default, and require an attribute set on service attributes or
>>>>> possibly even entity fields to say that restricted/safe HTML is allowed, or any HTML is allowed; this will break some things
>>>>> that actually need HTML initially, but fixing the broken things once found will be really easy
>>>>>
>>>>> 2. encode output: just in case HTML chars do get in somehow, we want to encode them so they are displayed literally and not
>>>>> interpreted as HTML by the browser; this will help avoid problems with messing up formatting when HTML is present, as well as
>>>>> helping with this security problem; this is easy to do in the various widgets (Screen, Form, Tree, Menu), and is tougher in
>>>>> FTL files if we want it encoded by default instead of manually using ?html on every field we want encoded, and I'd rather use
>>>>> the ESAPI encoder than the FTL one too; since much of this data is displayed right out of GenericValue objects, one possible
>>>>> solution is to change the GenericValue.get methods to do this encoding, and add a new get method that will not do encoding;
>>>>> this would handle the situations where the GenericValue is treated like a Map; this may also cause some crazy stuff to happen
>>>>> in places where gets are used in services and such and not in the UI... but I'm still thinking that through and am not sure if
>>>>> it will be a problem... it is kind of using bomb to swat a fly so collateral damage is likely
>>>>>
>>>>> 3. consider adding a token that is required for all requests in a session except the first one, use a constantly changing
>>>>> token, and have it added by the URL writing stuff based on a value in the session; this would change on every request, which
>>>>> is a pain because it means that any page in someone's browser other than the most recently rendered one would not work (a
>>>>> problem we have with the externalLoginKey stuff) unless we keep a certain number of the most recent tokens in the session and
>>>>> allow any of the last 10 or 20 or something to be used
>>>>>
>>>>> 4. related to #3, and relevant whether or not we do #3, add a unique token to all rendered forms and require that when
>>>>> processing the form input; if we only allow the tokens to be used once this also fits the common pattern used for eliminating
>>>>> accidental multiple submissions of the same form; this could be done easily with the Form Widget and the ServiceEventHandler
>>>>> (or perhaps all of the event handlers...), and more manually supported in other places like FTL forms; this would require some
>>>>> configuration, and again the annoying part is to cover as much as possible we would want this on by default which may cause
>>>>> problems for some things which would then need to changed to support it or disable it for that particular form and/or event
>>>>>
>>>>> ====================================
>>>>>
>>>>> I'm really interested in hearing what others have to say about these. Personally I've avoided most of these types of things
>>>>> because they always tend to cause a dozen problems for every problem they solve. I've mentioned some concerns, but there are
>>>>> many more. Some issues may just make the application less usable because of restrictions on being able to do things like use
>>>>> the back button (IMO supporting that is a critical part of any web app that is worth anything) or having a bunch of false
>>>>> positives for security errors because of some funny scenario that was not anticipated (and this isn't an if thing, it's a when
>>>>> and how often thing).
>>>>>
>>>>> -David
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Pierre Gaudin,
>>>> Consultant Fonctionnel Neogia, Apache-OFBiz
>>>> ERP en logiciel Libre
>>>>
>>>> Société Néréide
>>>> mobile : [phone]
>>>> bureau : [phone]
>>>> http://www.nereide.biz
>>>>
>>>
>>>
>>
>>
>

[David E Jones-3]

The problem with something really generic and independent like using  
ESAPI in this way is that it can't really leverage existing features  
and functionality of the framework.

There are various places where we'll want to normalize, validate, and  
encode data coming into or going out of the system. The ESAPI library  
has good tools for doing those tasks and while we may have to do other  
things, the first main task is to use that library in various parts of  
the framework to normalize and validate incoming data and encode  
outgoing data. The configuration for doing these things should be as  
integral to the framework as possible, ie reusing existing settings  
that are related and creating new ones (XML element attributes in  
entity or service or form or whatever defs) as needed. The general  
idea is that this becomes an "everyday" part of the framework and  
we're using ESAPI underneath to make it easier to implement, and to  
leverage the research and effort they've put into this.

If someone wanted to deploy the ESAPI servlet filter (ie in the  
web.xml file) and configure it independently as well then that is  
certainly an option. I don't want that to be the core approach because  
so much of that configuration would be redundant with things we  
already have and use, and would generally not be as automated.

-David


On Feb 2, 2009, at 9:41 AM, Philipp Hoppen wrote:

> Hi Pierre
>
> I took a look at the ESAPI library David suggested and tried to use  
> it in OFBiz for input filtering. It basically works as you described  
> by a filter specified in web.xml. There is a "SafeHTTPFilter" class  
> that uses a "SafeRequest" and a "SafeResponse" which call a  
> "Validator" object that is parametrized using regular expressions  
> defined in ESAPI.properties. The ESAPI reference implementations and  
> default property values are very restrictive (for example, leaving a  
> form parameter empty is not possible), so these have to be  
> customized for use in OFBiz. The good thing about a servlet filter  
> is that it is used very early in the processing and doesn't require  
> to modify code, but I don't know yet how we can make the link to an  
> entity or service field to allow HTML for specific parameters...
>
> A Validator can then probably be used for output encoding too (maybe  
> in GenericValue.get as David suggested).
>
> As we're interested in fixing these security issues in our OFBiz  
> projects (i made a Jira issue some weeks ago, https://issues.apache.org/jira/browse/OFBIZ-2121 
> , but it didn't get any attention), I can work at least 1 day per  
> week on this stuff. It would be nice if we could work on this  
> together in the same direction...
>
> --
> Philipp Hoppen,
> nowhow solutions AG, Laupenstrasse 1, CH-3008 Bern
> Phone [phone] http://www.nowhow.ch
>
>
> pierre schrieb:
>> Thank you Jacques,
>>
>> Other opinions on this approach?
>>
>> Can we work above?
>>
>> Pierre
>>
>> Jacques Le Roux wrote:
>>> Hi Pierre,
>>>
>>> From: "pierre" <[hidden email]>
>>>> Hi all,
>>>>
>>>> Here is a proposition on how to implement such XSS control:
>>>>
>>>> First we consider that all HHTP request should be filtering. So  
>>>> we could add a filter into web.xml for each webapp that replaces  
>>>> a set of dangerous characters by there HTML code.  By this way we  
>>>> can block all XSS attacks for entire application.
>>>
>>> Yes it makes sens indeed, that's what Michele also suggested in  
>>> this thread, (with less details) : http://www.nabble.com/Re%3A-Security-Issues-p21628377.html
>>>
>>>> After filtering all requests, we should add a way to parameterise  
>>>> this. So we could add 2 properties :
>>>>   - the first one to specifie a regex pattern that is used by  
>>>> filter engine
>>>>   - the second one to disable filtering
>>>>
>>>> And to be very flexible we can set those properties (or  
>>>> attributes) on 3 levels :
>>>>    - request (from request-map)
>>>>    - webapp (for a complete webbapp)
>>>>    - application (main level)
>>>
>>> The more flexible the better.
>>>
>>>> And finaly we could consider that if there are no paramateres on  
>>>> request level, then we look for webapp parameters. If there are  
>>>> no parameters on webapp we look for application parameters.
>>>>
>>>> By this way we could filter all request and set exeption or regex  
>>>> for a particular request or webb-app or entire application.
>>>>
>>>> What do you think about this.
>>>
>>> Yes this will cover this security aspect, and sounds good to me.
>>>
>>> Thanks
>>>
>>> Jacques
>>>
>>>> Pierre
>>>>
>>>> David E Jones wrote:
>>>>>
>>>>> Hello all.
>>>>>
>>>>> I'm actually a little surprised we're still where we are on  
>>>>> this, so I'm putting some time into this... understanding that  
>>>>> it will annoy as many people as it pleases (at first anyway...).
>>>>>
>>>>> In order to address various XSS and XSS-like security threats,  
>>>>> I'd like to get some real and comprehensive stuff in place.  
>>>>> Right now there are super-easy attacks that can be done, like  
>>>>> putting JavaScript in a field during checkout that gets executed  
>>>>> when a CSR (or anyone using the Order Manager) looks at the  
>>>>> order, or someone looks at it in the Party Manager or wherever.  
>>>>> That script can grab the session id and send it to a URL for  
>>>>> session hijacking, or it can directly perform some action like  
>>>>> marking the order as paid offline or creating a new admin  
>>>>> account or changing the users password or whatever. The script  
>>>>> could do anything the poor back-end user has access to do, and  
>>>>> that's just an example.
>>>>>
>>>>> The best issues on this are:
>>>>>
>>>>> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one,  
>>>>> good review of OFBiz security and applicable comments, good tips  
>>>>> to resolve)
>>>>> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/
>>>>> original one, including my silly comment on it)
>>>>>
>>>>> We have some simple code that does escaping for HTML chars, but  
>>>>> it's not really used anywhere. Anyway, I think we need something  
>>>>> more robust and comprehensive, especially given the fun ways of  
>>>>> getting around filters and other things presented here:
>>>>>
>>>>> http://ha.ckers.org/xss.html
>>>>>
>>>>> What I'd like to do is add the OWASP ESAPI library, which is BSD  
>>>>> licensed. There is a nice presentation about it as well here:
>>>>>
>>>>> http://code.google.com/p/owasp-esapi-java/
>>>>>
>>>>> and JavaDocs here:
>>>>>
>>>>> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>>>>>
>>>>> ======================================
>>>>>
>>>>> So, there's a tool, now how/where to use it in OFBiz...
>>>>>
>>>>> I think this will require a fair bit of work, and I know I'll  
>>>>> miss things that are important in this first pass, but we can do  
>>>>> some things to take care of the more obvious problems:
>>>>>
>>>>> 1. validate input: consider not allowing HTML in any field by  
>>>>> default, and require an attribute set on service attributes or  
>>>>> possibly even entity fields to say that restricted/safe HTML is  
>>>>> allowed, or any HTML is allowed; this will break some things  
>>>>> that actually need HTML initially, but fixing the broken things  
>>>>> once found will be really easy
>>>>>
>>>>> 2. encode output: just in case HTML chars do get in somehow, we  
>>>>> want to encode them so they are displayed literally and not  
>>>>> interpreted as HTML by the browser; this will help avoid  
>>>>> problems with messing up formatting when HTML is present, as  
>>>>> well as helping with this security problem; this is easy to do  
>>>>> in the various widgets (Screen, Form, Tree, Menu), and is  
>>>>> tougher in FTL files if we want it encoded by default instead of  
>>>>> manually using ?html on every field we want encoded, and I'd  
>>>>> rather use the ESAPI encoder than the FTL one too; since much of  
>>>>> this data is displayed right out of GenericValue objects, one  
>>>>> possible solution is to change the GenericValue.get methods to  
>>>>> do this encoding, and add a new get method that will not do  
>>>>> encoding; this would handle the situations where the  
>>>>> GenericValue is treated like a Map; this may also cause some  
>>>>> crazy stuff to happen in places where gets are used in services  
>>>>> and such and not in the UI... but I'm still thinking that  
>>>>> through and am not sure if it will be a problem... it is kind of  
>>>>> using bomb to swat a fly so collateral damage is likely
>>>>>
>>>>> 3. consider adding a token that is required for all requests in  
>>>>> a session except the first one, use a constantly changing token,  
>>>>> and have it added by the URL writing stuff based on a value in  
>>>>> the session; this would change on every request, which is a pain  
>>>>> because it means that any page in someone's browser other than  
>>>>> the most recently rendered one would not work (a problem we have  
>>>>> with the externalLoginKey stuff) unless we keep a certain number  
>>>>> of the most recent tokens in the session and allow any of the  
>>>>> last 10 or 20 or something to be used
>>>>>
>>>>> 4. related to #3, and relevant whether or not we do #3, add a  
>>>>> unique token to all rendered forms and require that when  
>>>>> processing the form input; if we only allow the tokens to be  
>>>>> used once this also fits the common pattern used for eliminating  
>>>>> accidental multiple submissions of the same form; this could be  
>>>>> done easily with the Form Widget and the ServiceEventHandler (or  
>>>>> perhaps all of the event handlers...), and more manually  
>>>>> supported in other places like FTL forms; this would require  
>>>>> some configuration, and again the annoying part is to cover as  
>>>>> much as possible we would want this on by default which may  
>>>>> cause problems for some things which would then need to changed  
>>>>> to support it or disable it for that particular form and/or event
>>>>>
>>>>> ====================================
>>>>>
>>>>> I'm really interested in hearing what others have to say about  
>>>>> these. Personally I've avoided most of these types of things  
>>>>> because they always tend to cause a dozen problems for every  
>>>>> problem they solve. I've mentioned some concerns, but there are  
>>>>> many more. Some issues may just make the application less usable  
>>>>> because of restrictions on being able to do things like use the  
>>>>> back button (IMO supporting that is a critical part of any web  
>>>>> app that is worth anything) or having a bunch of false positives  
>>>>> for security errors because of some funny scenario that was not  
>>>>> anticipated (and this isn't an if thing, it's a when and how  
>>>>> often thing).
>>>>>
>>>>> -David
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Pierre Gaudin,
>>>> Consultant Fonctionnel Neogia, Apache-OFBiz
>>>> ERP en logiciel Libre
>>>>
>>>> Société Néréide
>>>> mobile : [phone]
>>>> bureau : [phone]
>>>> http://www.nereide.biz
>>>>
>>>
>>>
>>
>>
>

[David E Jones-3]

With my latest commit (rev 742352) I've finished a first pass on a few  
things for this. My commit log for that is included below (and has  
some important notes).

Now in place are:

1. canonicalization for all incoming parameters
2. encoding of output for all String fields in forms, and for all  
String objects in FTL files (use something other than a String object,  
like a StringBuffer or something if you don't want encoding)
3. validation of input in the Service Engine to restrict allowed HTML,  
with three options for the allow-html attribute including none  
(default), safe (antisamy-based), and any (no checking of HTML input)

Which these in place we protect from XSS and related attacks in a few  
ways, and most data should be validated in input and encoded on output  
so that it is not interpreted by the browser as HTML. This is also a  
general good thing as users won't accidently type in characters that  
mess up the HTML either.

There is still potential for security holes, especially when the main  
(ie best practice recommended) OFBiz tools are not used. Just using  
the ControlServlet, the Screen Widget and the Service Engine should  
handle most of these issues now (unless someone explicitly disables  
these tools).

There are also a bunch of places still where HTML should be allowed,  
at least the AntiSamy "safe" HTML, but currently isn't.

I've spent dozens of hours on this stuff now and done a LOT of  
testing, but I know there is a lot more testing and refinement to do.  
We could use help from anyone listening with the following things:

1. anything that used to work but now doesn't, especially funny text  
on screens or complaints about HTML related things
2. security vulnerability tests: now we want to hit the public facing  
(ecommerce, cmssite, etc) apps and the back-end apps to check as many  
vulnerabilities as we can
3. requests for loosening up in certain places that are now by default  
too strict (probably because they haven't been reviewed yet with these  
new defaults in place)

Help with these things would be greatly appreciated! Also, thanks for  
the help so far from those who have commented and been patient and such.

Going forward, some things I still want to work on are dynamic session  
tokens to use in addition to a jsessionid in order to prevent session  
hijacking, and also form submission tokens to avoid form hijacking  
(just in case XSS slips in, etc) and also to avoid duplicate  
submission of forms. As usual when these happen is another question!  
Things have been piling up as I pushed on these... :(

-David


==================================
Date: Mon Feb  9 09:34:34 2009
New Revision: 742352

URL: http://svn.apache.org/viewvc?rev=742352&view=rev
Log:
Added new allow-html tag on the attribute, auto-attribute, and  
override elements; has 3 options: none, safe, and any; the comments in  
the XSD file describe what each of these do; the important thing to  
know is that none is the default meaning no html is allowed; if html  
is needed use safe and look at the antisamy-esapi.xml file to see  
policy details; in extreme trust cases use any where any html is  
allowed; note that many services need updating which should allow at  
least safe html, and it may take some time to discover all of those  
and get them handled; please send in issues and requests for service  
attributes that should allow safe html



On Jan 23, 2009, at 3:41 AM, David E Jones wrote:

>
> Hello all.
>
> I'm actually a little surprised we're still where we are on this, so  
> I'm putting some time into this... understanding that it will annoy  
> as many people as it pleases (at first anyway...).
>
> In order to address various XSS and XSS-like security threats, I'd  
> like to get some real and comprehensive stuff in place. Right now  
> there are super-easy attacks that can be done, like putting  
> JavaScript in a field during checkout that gets executed when a CSR  
> (or anyone using the Order Manager) looks at the order, or someone  
> looks at it in the Party Manager or wherever. That script can grab  
> the session id and send it to a URL for session hijacking, or it can  
> directly perform some action like marking the order as paid offline  
> or creating a new admin account or changing the users password or  
> whatever. The script could do anything the poor back-end user has  
> access to do, and that's just an example.
>
> The best issues on this are:
>
> https://issues.apache.org/jira/browse/OFBIZ-1959 (newer one, good  
> review of OFBiz security and applicable comments, good tips to  
> resolve)
> https://issues.apache.org/jira/browse/OFBIZ-260 (the old/original  
> one, including my silly comment on it)
>
> We have some simple code that does escaping for HTML chars, but it's  
> not really used anywhere. Anyway, I think we need something more  
> robust and comprehensive, especially given the fun ways of getting  
> around filters and other things presented here:
>
> http://ha.ckers.org/xss.html
>
> What I'd like to do is add the OWASP ESAPI library, which is BSD  
> licensed. There is a nice presentation about it as well here:
>
> http://code.google.com/p/owasp-esapi-java/
>
> and JavaDocs here:
>
> http://owasp-esapi-java.googlecode.com/svn/trunk_doc/index.html
>
> ======================================
>
> So, there's a tool, now how/where to use it in OFBiz...
>
> I think this will require a fair bit of work, and I know I'll miss  
> things that are important in this first pass, but we can do some  
> things to take care of the more obvious problems:
>
> 1. validate input: consider not allowing HTML in any field by  
> default, and require an attribute set on service attributes or  
> possibly even entity fields to say that restricted/safe HTML is  
> allowed, or any HTML is allowed; this will break some things that  
> actually need HTML initially, but fixing the broken things once  
> found will be really easy
>
> 2. encode output: just in case HTML chars do get in somehow, we want  
> to encode them so they are displayed literally and not interpreted  
> as HTML by the browser; this will help avoid problems with messing  
> up formatting when HTML is present, as well as helping with this  
> security problem; this is easy to do in the various widgets (Screen,  
> Form, Tree, Menu), and is tougher in FTL files if we want it encoded  
> by default instead of manually using ?html on every field we want  
> encoded, and I'd rather use the ESAPI encoder than the FTL one too;  
> since much of this data is displayed right out of GenericValue  
> objects, one possible solution is to change the GenericValue.get  
> methods to do this encoding, and add a new get method that will not  
> do encoding; this would handle the situations where the GenericValue  
> is treated like a Map; this may also cause some crazy stuff to  
> happen in places where gets are used in services and such and not in  
> the UI... but I'm still thinking that through and am not sure if it  
> will be a problem... it is kind of using bomb to swat a fly so  
> collateral damage is likely
>
> 3. consider adding a token that is required for all requests in a  
> session except the first one, use a constantly changing token, and  
> have it added by the URL writing stuff based on a value in the  
> session; this would change on every request, which is a pain because  
> it means that any page in someone's browser other than the most  
> recently rendered one would not work (a problem we have with the  
> externalLoginKey stuff) unless we keep a certain number of the most  
> recent tokens in the session and allow any of the last 10 or 20 or  
> something to be used
>
> 4. related to #3, and relevant whether or not we do #3, add a unique  
> token to all rendered forms and require that when processing the  
> form input; if we only allow the tokens to be used once this also  
> fits the common pattern used for eliminating accidental multiple  
> submissions of the same form; this could be done easily with the  
> Form Widget and the ServiceEventHandler (or perhaps all of the event  
> handlers...), and more manually supported in other places like FTL  
> forms; this would require some configuration, and again the annoying  
> part is to cover as much as possible we would want this on by  
> default which may cause problems for some things which would then  
> need to changed to support it or disable it for that particular form  
> and/or event
>
> ====================================
>
> I'm really interested in hearing what others have to say about  
> these. Personally I've avoided most of these types of things because  
> they always tend to cause a dozen problems for every problem they  
> solve. I've mentioned some concerns, but there are many more. Some  
> issues may just make the application less usable because of  
> restrictions on being able to do things like use the back button  
> (IMO supporting that is a critical part of any web app that is worth  
> anything) or having a bunch of false positives for security errors  
> because of some funny scenario that was not anticipated (and this  
> isn't an if thing, it's a when and how often thing).
>
> -David
>
>

[euronymous]

David E Jones-3 wrote

2. security vulnerability tests: now we want to hit the public facing  
(ecommerce, cmssite, etc) apps and the back-end apps to check as many  
vulnerabilities as we can

In reply to your find-bug-campaing:

https://issues.apache.org/jira/browse/OFBIZ-1959

See my latest comment. A reflected XSS in latest trunk (partymgr --> viewprofile --> partyId).

Let me know David

Anyway, really thanks for the time you spent implementing esapi and antysamy.

All the best


Michele Orrù

[David E Jones-3]

On Feb 18, 2009, at 4:25 AM, euronymous wrote:

>
>
>
> David E Jones-3 wrote:
>>
>>
>> 2. security vulnerability tests: now we want to hit the public facing
>> (ecommerce, cmssite, etc) apps and the back-end apps to check as many
>> vulnerabilities as we can
>>
>>
>
> In reply to your find-bug-campaing:
>
> https://issues.apache.org/jira/browse/OFBIZ-1959
>
> See my latest comment. A reflected XSS in latest trunk (partymgr -->
> viewprofile --> partyId).
>
> Let me know David

I'll try to look at that in the next day or two. It is probably a  
place that doesn't uses the common tools and so gets around these  
somehow...

-David

[euronymous]

David E Jones-3 wrote

I'll try to look at that in the next day or two. It is probably a  
place that doesn't uses the common tools and so gets around these  
somehow...

David

I'm asking you a favour :)

I'm analyzing all about your ESAPI/AntiSamy impementation.
Let me understand better: all the files where you put your changes/integrations
are traced in revisions 741857 and 742352?

Let me know if I'm missing some classes that are not listed in these two commits:
I'm really interested about knowing exactly where did you put the code, to better
understand Ofbiz internal architecture and how did you integrate esapi.

Thank you David

All the best


Michele Orrù

[David E Jones-3]

On Feb 20, 2009, at 8:37 AM, euronymous wrote:

> David E Jones-3 wrote:
>> I'll try to look at that in the next day or two. It is probably a
>> place that doesn't uses the common tools and so gets around these
>> somehow...
>
> David
>
> I'm asking you a favour :)
>
> I'm analyzing all about your ESAPI/AntiSamy impementation.
> Let me understand better: all the files where you put your
> changes/integrations
> are traced in revisions 741857 and 742352?
>
> Let me know if I'm missing some classes that are not listed in these  
> two
> commits:
> I'm really interested about knowing exactly where did you put the  
> code, to
> better
> understand Ofbiz internal architecture and how did you integrate  
> esapi.

There are more commits than that. The easiest place to see them is  
probably FishEye:

http://fisheye6.atlassian.com/changelog/~author=jonesde/ofbiz/

You'll need to look back to 6 Feb, rev 741442, and as far forward as  
10 Feb, rev 742866. Those two commit and most in between them are  
related to the canonicalization, HTML filtering/validation, and output  
encoding.

-David