Dealing with ESAPI in CMS

[byersa]

I guess one of the first places that we will run into ESAPI affects is in
content management when we try to store images and the like that are larger
than the default "maxInputSize" = 5000 specified in the antisamy-esapi.xml
file.

What would be the best approach to dealing with this?

1. Just modify the maxInputSize value? I may want to limit files that are
uploaded, but not content that is persisted some other way.

2. In that case do I override the createDataText service (which would mean
overriding the "createTextContent" service)?

Are there any other options I am missing like specifically changing the
value of allowHTML (if that were set to "any" there would be no content
limits, right?) for a specific service call?

Thanks,
-Al

[David E Jones-3]

Could you be more specific? In other words, which part of the  
application were you using and what was the error message that you got?

You mentioned a problem uploading images... which baffles me the most  
because the ESAPI changes are _only_ for String attributes on  
services. What was the error that you got for that?

More details would be really helpful about the specific issue you're  
running into. Beyond that as a generality (possibly not related to  
what you're running into?), I agree that we should leave the default  
pretty high, and I've just committed a change that does that.

-David


On Feb 23, 2009, at 4:52 PM, Al Byers wrote:

> I guess one of the first places that we will run into ESAPI affects  
> is in
> content management when we try to store images and the like that are  
> larger
> than the default "maxInputSize" = 5000 specified in the antisamy-
> esapi.xml
> file.
>
> What would be the best approach to dealing with this?
>
> 1. Just modify the maxInputSize value? I may want to limit files  
> that are
> uploaded, but not content that is persisted some other way.
>
> 2. In that case do I override the createDataText service (which  
> would mean
> overriding the "createTextContent" service)?
>
> Are there any other options I am missing like specifically changing  
> the
> value of allowHTML (if that were set to "any" there would be no  
> content
> limits, right?) for a specific service call?
>
> Thanks,
> -Al

[byersa]

I was being too general. I was not uploading images. I just used
createTextContent and it failed because the limit was set too low. I saw how
to fix that with maxInputSize and was just asking the question if there were
more granular ways to deal with the problem.

-Al

On Mon, Feb 23, 2009 at 5:30 PM, David E Jones
<[hidden email]>wrote:

>
> Could you be more specific? In other words, which part of the application
> were you using and what was the error message that you got?
>
> You mentioned a problem uploading images... which baffles me the most
> because the ESAPI changes are _only_ for String attributes on services. What
> was the error that you got for that?
>
> More details would be really helpful about the specific issue you're
> running into. Beyond that as a generality (possibly not related to what
> you're running into?), I agree that we should leave the default pretty high,
> and I've just committed a change that does that.
>
> -David
>
>
>
> On Feb 23, 2009, at 4:52 PM, Al Byers wrote:
>
>  I guess one of the first places that we will run into ESAPI affects is in
>> content management when we try to store images and the like that are
>> larger
>> than the default "maxInputSize" = 5000 specified in the antisamy-esapi.xml
>> file.
>>
>> What would be the best approach to dealing with this?
>>
>> 1. Just modify the maxInputSize value? I may want to limit files that are
>> uploaded, but not content that is persisted some other way.
>>
>> 2. In that case do I override the createDataText service (which would mean
>> overriding the "createTextContent" service)?
>>
>> Are there any other options I am missing like specifically changing the
>> value of allowHTML (if that were set to "any" there would be no content
>> limits, right?) for a specific service call?
>>
>> Thanks,
>> -Al
>>
>
>

[David E Jones-3]

Great, cool. Hopefully the new size (100000) should fix that for you.  
Thanks for reporting it, and of course if you run into anything else  
please do write it up again. I'm trying to keep an eye on this stuff  
in particular and be around to fix things since this sort of low-level  
change tends to impact a lot and causes random problems all over... :(

-David


On Feb 23, 2009, at 9:29 PM, Al Byers wrote:

> I was being too general. I was not uploading images. I just used
> createTextContent and it failed because the limit was set too low. I  
> saw how
> to fix that with maxInputSize and was just asking the question if  
> there were
> more granular ways to deal with the problem.
>
> -Al
>
> On Mon, Feb 23, 2009 at 5:30 PM, David E Jones
> <[hidden email]>wrote:
>
>>
>> Could you be more specific? In other words, which part of the  
>> application
>> were you using and what was the error message that you got?
>>
>> You mentioned a problem uploading images... which baffles me the most
>> because the ESAPI changes are _only_ for String attributes on  
>> services. What
>> was the error that you got for that?
>>
>> More details would be really helpful about the specific issue you're
>> running into. Beyond that as a generality (possibly not related to  
>> what
>> you're running into?), I agree that we should leave the default  
>> pretty high,
>> and I've just committed a change that does that.
>>
>> -David
>>
>>
>>
>> On Feb 23, 2009, at 4:52 PM, Al Byers wrote:
>>
>> I guess one of the first places that we will run into ESAPI affects  
>> is in
>>> content management when we try to store images and the like that are
>>> larger
>>> than the default "maxInputSize" = 5000 specified in the antisamy-
>>> esapi.xml
>>> file.
>>>
>>> What would be the best approach to dealing with this?
>>>
>>> 1. Just modify the maxInputSize value? I may want to limit files  
>>> that are
>>> uploaded, but not content that is persisted some other way.
>>>
>>> 2. In that case do I override the createDataText service (which  
>>> would mean
>>> overriding the "createTextContent" service)?
>>>
>>> Are there any other options I am missing like specifically  
>>> changing the
>>> value of allowHTML (if that were set to "any" there would be no  
>>> content
>>> limits, right?) for a specific service call?
>>>
>>> Thanks,
>>> -Al
>>>
>>
>>

[byersa]

I am storing a very large JSON string in the database using the CMS. Am I
right in understanding that because the createTextContent service does not
have an "allowHtml" attribute on the textData field set to "none" that in
ModelService.validate method it is the
StringUtil.checkStringForHtmlStrictNone call that is encoding the double
quotes that are in the string?

What would setting allowHtml to "safe" do? Still encode?

If this is the case, do we have any options other than writing different
versions of content persisting services to handle the case where we do not
want encoding to happen?

-Al

[David E Jones-3]

For the most part we'd only want to accept "safe" HTML, but it is  
certainly conceivable to need something more open/flexible/etc. It  
might be good to have two service defs (both can call the same service  
impl), with names to denote the difference, ie a suffix of "SafeHtml"  
and "AnyText" for the other.

As for storing JSON strings... it sounds a bit odd. The Content stuff  
is really meant for content that is written and read by humans. I do  
agree the above change would be good, but perhaps there is also a  
better place to store your JSON strings (ie it sounds a little bit  
"hackish"... ;) ).

-David


On Feb 23, 2009, at 10:20 PM, Al Byers wrote:

> I am storing a very large JSON string in the database using the CMS.  
> Am I
> right in understanding that because the createTextContent service  
> does not
> have an "allowHtml" attribute on the textData field set to "none"  
> that in
> ModelService.validate method it is the
> StringUtil.checkStringForHtmlStrictNone call that is encoding the  
> double
> quotes that are in the string?
>
> What would setting allowHtml to "safe" do? Still encode?
>
> If this is the case, do we have any options other than writing  
> different
> versions of content persisting services to handle the case where we  
> do not
> want encoding to happen?
>
> -Al

[byersa]

On Tue, Feb 24, 2009 at 1:54 AM, David E Jones
<[hidden email]>wrote:

>
> For the most part we'd only want to accept "safe" HTML, but it is certainly
> conceivable to need something more open/flexible/etc. It might be good to
> have two service defs (both can call the same service impl), with names to
> denote the difference, ie a suffix of "SafeHtml" and "AnyText" for the
> other.
>
> As for storing JSON strings... it sounds a bit odd. The Content stuff is
> really meant for content that is written and read by humans. I do agree the
> above change would be good, but perhaps there is also a better place to
> store your JSON strings (ie it sounds a little bit "hackish"... ;) ).

That is an interesting thought and it would solve my problem. The JSON
string is a complete representation of an online test (takes too long to
build it on-demand), but it will not be read directly by humans. Thanks
again for a simple and pragmatic solution to my problem and shedding new
light on CMS philosophy.

-Al

>
> -David
>
>
>
> On Feb 23, 2009, at 10:20 PM, Al Byers wrote:
>
>  I am storing a very large JSON string in the database using the CMS. Am I
>> right in understanding that because the createTextContent service does not
>> have an "allowHtml" attribute on the textData field set to "none" that in
>> ModelService.validate method it is the
>> StringUtil.checkStringForHtmlStrictNone call that is encoding the double
>> quotes that are in the string?
>>
>> What would setting allowHtml to "safe" do? Still encode?
>>
>> If this is the case, do we have any options other than writing different
>> versions of content persisting services to handle the case where we do not
>> want encoding to happen?
>>
>> -Al
>>
>
>