Unable to upload large files

[kipz]

When uploading large files, a stream is passed to the API:

Flickr.getUploader().upload(InputStream, params)

However, REST.java buffers everything in to a ByteArrayOutputStream before passing it to the scribe OAuth library. Therefore, uploading large files can often cause out of memory errors (depends on the size of file and heap of course).

Scribe OAuth (as far as I know) doesn't support any streaming yet, so we could either create our own implementation, swap out for an implementation that does support this, or ask and wait for those guys to fix theirs.

I've opened an issue on Scribe related to this:

scribejava/scribejava#347

[kipz]

I forgot to mention, that this currently prevents me from uploading videos (MOVs) over 100MB with a default heap with Oracle JRE 7 on Mac.

[boncey]

I don't use Flickr4Java for video (yet) so I've not seen this.

I guess we can see what the Scribe folks say as that would be the best solution I think.
If they request a patch is that something you'd consider doing?

[kipz]

I'll have a go, but to be honest, doing all that on top of HTTPConnection will be a bit of a bore. Ideally they would integrate Apache HTTP client (or similar) which would give much of it for free.

[boncey]

Good point.
This pull request looks interesting, wonder if it will get accepted.
scribejava/scribejava#250

[kipz]

Let's hope so, but it's looking pretty quiet over there at the moment...