How can I Upload Larger Files?

As many users continue to have difficulties related to file size while uploading files with jDownloads, I decided to try to shed some light on the difference between the upload settings in your jDownloads backend and the global configuration settings of the webserver where your site is hosted. In a nutshell, the upload size settings you make in your jDownloads backend are subject to the limits imposed by your webserver. This means, for example, that if you set a maximum file size for uploads in the jDownloads Configuation - Uploads that is say 10Mb and the maximum file size in your webservers global settings is 6Mb then the actual limit is 6Mb.  That is the maximum upload size is the smaller of the two settings.

Like most other Joomla! components, the majority of jDownloads code is written in PHP, which means the code executes on your webserver. This works quite differently from a client side Java applet, which excutes on your machine after it was  downloaded together with the page you requested to view. Hence, you may easily influence the behavior of Java based script, for example by enabling or disabling options in the configuration of your web browser. This isn't at all the case with PHP:  everything runs on the webserver; PHP configuration settings can only be changed there; and then only if you have administrative access to that server. On some servers there may be additional setting which are webserver specific. At the present time we are aware of an extra non PHP related limit on Apache based servers. If your site is using IIS based (Microsoft) server then see the IIS Server section below as there is a known limitation of about 28M.  This may be raised.

Please note, by the way, that this has nothing to do with your Joomla! backend, nor with your access to the control panel of your hosting account. Administrative access, or root access, to the global configuration settings of a webserver is often reserved to the people who own and/or professionally administer it, which in most cases is your webhosting service provider.

PHP configuration settings are within a file called php.ini, which is located on the webserver. The exact location of this file is displayed under System >> System Info >> PHP Info >> Configuration File php.ini path.

As I said earlier, you can only lay your hands upon this file if you have administrative access. In practice, this means that it's entirely out of reach for you. You may, however, view the current settings made within php.ini.
If you have Joomla!, a quick way to do this is to look them up in your backend under System >> System Info >> PHP Info.

Alternatively, you may also create a simple text file, call it info.php (or the likes), and place the following lines into it:
<?php
phpinfo();
?>

Then, upload this file to your server and point your browser to it.  For example suppose you call your file myinfo.php and place it in directory called test, then give your browser the command http://www.yoursite.com/test/myinfo.php  The result is only slightly different from the info displayed in your Joomla! backend. In both cases, you'll be able to examine the specific settings made by the system administrator in regard to PHP.

Note Never leave the file with the phpinfo() comand on your site.  It gives valuable info to hackers!

This brings us back to jDownloads: no matter which upload setting you chose in your jDownloads configuration panel, all will be executed by PHP and hence ultimately obey the global PHP settings made by the webserver system administrator. Many of such general settings exist, and they often influence one another as well.

Before I go into detail on the particular parameters related to file upload, another important note:
The jDownloads configuration manager provides you with an option to determine user rights in regard to the size and the amount of files they may upload. Please note that these settings exclusively regard the jDownloads frontend-upload, and their only purpose is to allow you to place limitations on whatever is allowed in your servers php.ini.

Hence, you won't be able to expect a setting for a a very large file size you make in your jDownloads backend to have an effect unless the settings inside php.ini allow for it. If the settings within your servers php.ini determine a maximum file size for uploads which is smaller than the one you entered in your jDownloads backend, the setting in php.ini is the one that counts.

Now, the php-ini settings regarding upload:

  • file_uploads

file_uploads determines whether uploads can be made to the server via HTTP or not. If this is set to OFF, no uploads of any kind are allowed.

  • upload_max_filesize

upload_max_filesize limits the maximum size of an individual file be uploaded. Normally, this value is entered in MB (Megabyte). During FTP upload this value concerns the size of each individual file. During batch upload, however, this value concerns the size of the entire zip file you upload, which is where limitations can make themselves more easily apparent.

  • post_max_size

upload_max_filesize is not the only value that determines the maximum size of a file to be uploaded. post_max_size is among the settings that influence upload_max_filesize. It determines the maximum size of POST files. Usually, file uploads are POST files. To allow the upload of larger files, the value given in post_max_size must be larger than the value in upload_max_filesize.

  • memory_limit

memory_limit determines the maximum amount of memory that a script may take up during an operation. The value is usually given in bytes. This value should be larger than the one given in post_max_size. A value of -1 means that there are no limitatons. As the exact amount of memory used by a script during upload is difficult to predict, memory_limit is a setting to watch out for.The above three settings are essential in configuring file uploads.
Their relation should always be as such:

memory_limit >> post_max_size > upload_max_filesize

If upload_max_filesize was set to 2 MB, a batch upload of 4MB zip file for example would simply be bound to fail.

These 3 settings, however, are not the only ones that determine a smooth functioning of your file uploads. Another important setting, well capable of ruining your uploads even if correct values were set for upload_max_files, post_max_size and memory_limit, is this:

  • max_execution_time

max_execution_time defines the maximum time (in seconds, or rather miliseconds!) that a script is allowed to take up in order to perform an operation. If it takes any longer, the parser (=server side software that interpretates the script in accordance to the current settings) will halt the execution. This is a critical matter especially for uploads, as upload and extraction of a large zipfile, image copying and resizing etc. all take their time.

Add a slow internet connection to this, and the end is in sight rather quickly. jDownloads will go throught the tasks until it hits the time limit set in php.ini and then stops. This is why sometimes only a part of zip file is processed during a batch upload with jDownloads: The time limit has been reached and the parser has halted the execution of script. We are currently looking for ways to avoid such undesired interruptions, but the matter isn't a trivial one.

Server Settings

PHP settings are not the only settings on a server that can have an impact on your uploads. Even with correct values for the settings we've just discussed, server configuration parameters that rank even higher than PHP settings may interfere. You'll know that this was the case when the error message you see is no longer issued by PHP but by another application, most likely Apache, the webserver software itself. It shouldn't come as a surprise that access to base settings is even more restricted than access to PHP settings, and that hosts are rarely willing to make changes to them. It's worth noting here that host related issues of that sort would be of a rather general nature, and would cause problems not with one particular PHP script but many others as well.

  • Apache Settings

At the time of writing we are unaware of any extra limits placed on uploads by the Apache based servers.

  • IIS Servers

We have recently become aware of a limitation on uploads set in the IIS based servers - many thanks to user Wahone for bringing this to our attention.

In an  IIS 7.0+ server  there is a limit called maxAllowedContentLength that has a default value of 28.6 Mbyte. Earlier IIS versions had a much lower limit and thge variable concerned is maxRequestLength.  Uploading files greater than that may result in a 403 or 404 error, a system reset message, or similar. The actual error message depends on which browesr you are using.

There are three ways to fix the problem - see the links below for more details. The first link shows all three ways; the second one has a good description of one method; the third link details some other additional potential 'challenges'.

Alternatively it may be better to contact your webhosting service provider!

 

Final Remarks
I hope this text helped to add some clarity regardiing the the processes involved in the upload functionality of jDownloads. Unfortunately, many factors are at play, and not all, hardly any infact, lie within the reach or the responsibility of jDownloads.

When you run into problems, keep in mind that you can also upload your files via FTP. With the jDownloads auto monitoring function, you can add easy the new uploaded folders or files in the jDownloads database tables. 

As many users continue to have difficulties related to file size while uploading images with PonyGallery ML, I decided to try to shed some light on the difference between the upload settings in your PonyGallery ML backend and the global configuration settings of the webserver where your site is hosted. In a nutshell, the settings you make in your PonyGallery ML backend always obey those of the webserver. This means, for example, that you won't be able to set a maximum file size for uploads that is larger than the value for maximum file size in your webservers global settings.

Like most other Joomla! components, most PonyGallery ML code was written in PHP. PHP is a server side application, which means that all functions of a PHP script are executed on the server where your web pages are located. This works quite differently from a client side Java applet, which excutes on your machine after it was is downloaded together with the page you requested to view. Hence, you may easily influence the behavior of Java based script, for example by enabling or disabling options in the configuration of your web browser. This isn't at all the case with PHP: As everything runs on the webserver, PHP configuration settings can only be changed there, and only so if you have administrative access to that server.

Please note, by the way, that this has nothing to do with your Joomla! backend, nor with your access to the control panel of your hosting account. Administrative access, or root access, to the global configuration settings of a webserver is reserved to the people who own and/or professionally administer it (= in most cases, your webhost).
 
PHP configuration settings are within a file called php.ini, which is located on the webserver. The exact location of this file is displayed under System >> System Info >> PHP Info >> Configuration File php.ini path.
 
As I said earlier, you can only lay your hands upon this file if you have administrative access. In practice, this means that it's entirely out of reach for you. You may, however, view the current settings made within php.ini.
If you have Joomla!, a quick way to do this is to look them up in your backend under System >> System Info >> PHP Info.

Alternatively, you may also create a simple text file, call it info.php (or the likes), and place the following lines into it:

<?php
 phpinfo();
?>

Then, upload this file to your server and point your browser to it. The result is only slightly different from the info displayed in your Joomla! backend. In both cases, you'll be able to examine the specific settings made by the system administrator in regard to PHP....

 .... which brings us back to PonyGallery ML: No matter which upload setting you chose in your PonyGallery ML configuration panel, all will be executed by PHP and hence ultimately obey the global settings PHP settings made by the system administrator. Many of such general settings exist, and they often influence one another as well.
 

Before I go into detail on the particular parameters related to file upload, another important note:
The PonyGallery ML configuration manager provides you with an option to determine user rights in regard to the size and the amount of images they may upload. Please note that these settings exclusively regard the PonyGallery ML frontend-upload, and their only purpose is to allow you to place limitations on whatever is allowed in your servers php.ini.

Hence, you won't be able to expect a setting for a a very large file size you make in your PonyGallery ML backend to have an effect unless the settings inside php.ini allow for it. If the settings within your servers php.ini determine a maximum file size for uploads which is smaller than the one you entered in your PonyGallery ML backend, the setting in php.ini is the one that counts.



Now, the php-ini settings regarding upload:
 


file_uploads


file_uploads determines whether uploads can be made to the server via HTTP or not. If this is set to OFF, no uploads of any kind are allowed.

 

upload_max_filesize

upload_max_filesize limits the maximum size of an individual file be uploaded. Normally, this value is entered in MB (Megabyte). During FTP upload or the upload of single images through PGML, this value concerns the size of each individual file. During batch upload, however, this value concerns the size of the entire zip file you upload, which is where limitations can make themselves more easily apparent.

 

post_max_size

upload_max_filesize is not the only value that determines the maximum size of a file to be uploaded. post_max_size is among the settings that influence upload_max_filesize. It determines the maximum size of POST files. Usually, file uploads are POST files. To allow the upload of larger files, the value given in post_max_size must be larger than the value in upload_max_filesize.

 

memory_limit

memory_limit determines the maximum amount of memory that a script may take up during an operation. The value is usually given in bytes. This value should be larger than the one given in post_max_size. A value of -1 means that there are no limitatons. As the exact amount of memory sed by a script during upload is difficult to predict, memory_limit is a setting to watch out for.


An approximate of the minimal use of virtual memory by Ponygallery ML (or the GL library, respectively) during the resizing of the images can be derived by using the following formula:

image width in pixel * image height in pixel * bit depth in byte = required memory in byte

As you see, memory load is determined by the dimensions of the image and its bit depth, rather than by the space it takes up on your hard disk. Say we had the server resize an image of  2272 x 1704 px and a depth of 24 bit (24 bit = 3 bytes), then the result would turn out to be 11 MB of required memory (2272 x 1704 x 3 = 11.614.464 bytes). That's for the resizing alone, to which we'd still need to add the memory required for the Ponygallery ML scripts (appr.8 MB.). This already gives us around 20 MB alltogether, and if  memory_limit  has been set to 16 MB, that's the end of the line for us already.




The above 3 settings are essential in configuring  file upload.
Their relation should always be as such:

memory_limit >> post_max_size > upload_max_filesize

If upload_max_filesize was set to 2 MB, a batch upload of 4MB zip file for example would simply be bound to fail.

These 3 settings, however, are not the only ones that determine a smooth functioning of your file uploads. Another important setting, well capable of ruining your uploads even if correct values were set for upload_max_files, post_max_size and memory_limit, is this:



max_execution_time


max_execution_time defines the maximum time (in seconds, or rather miliseconds!) that a script is allowed to take up in order to perform an operation. If it takes any longer, the parser (=server side software that interpretates the script in accordance to the current settings) will halt the execution. This is a critical matter especially for uploads, as upload and extraction of a large zipfile, image copying and resizing etc. all take their time.

Add a slow internet connection to this, and the end is in sight rather quickly. Ponygallery ML will geo throught the tasks until it hits the time limit set in php.ini and then stops. This is why sometimes only a part of zip file is processed during a batch upload with Ponygallery ML: The time limit has been reached and the parser has halted the execution of script. We are currently looking for ways to avoid such undesired interruptions, but the matter isn't a trivial one.





Apache Settings

PHP settings are not the only settings on a server that can have an impact on your uploads. Even with correct values for the settings we've just discussed, server configuration parameters that rank even higher than PHP settings may interfere. You'll know that this was the case when the error message you see is no longer issued by PHP but by another application, most likely Apache, the webserver software itself. It shouldn't come as a surprize that access to Apache settings is even more restricted than access to PHP settings, and that hosts are rarely willing to make changes to them.

It's worth noting here that host related issues of that sort would be of a rather general nature, and would cause problems not with one particular PHP script but many others as well. Typically, on a host where uploads with Ponygallery ML chronically fail in spite of correct or even generous PHP settings, they will also fail with ICE Gallery, other PHP galleries, or other scipts that upload and process large files (which rules out Ponygallery ML as a principal culprit).





I hope this text helped to add some clarity regardiing the the processes involved in the upload functionality of Ponygallery ML. Unfortunately, many factors are at play, and not all -hardly any, infact- lie within the reach or the responsibility of Ponygallery ML.

When you run into problems, keep in mind that FTP upload mode of Ponygallery ML circumvents most of the difficulties that can be imposed by php.ini settings (with the exception of max_execution_time).
 


Of course there are yet other factors out there that may create problems, such as wwwrun, but we'll talk about these another time.

 ColinM- Updated August 2017

  • Monday, 16 June 2014