Let's consider the following situation: You have built a file repository system or a photo album in CakePHP and you would like the user to download all the files or photos in a specific folder in one convenient zip file. In principle, you have several options how to proceed:
Here, I will discuss why the latter option is most likely the best and how to implement this in CakePHP.
Without any doubt, the first two options require both computational time and disk space. If you pre-generate the zip file, this means that for each file or photo you have added, some time is spent adding the item to the zip file. Consequently, uploading of new content will take more time. Moreover, you will need almost 1.5x the normal amount of storage space, as you need to store both the raw images as well as the zip file.
Only generating the zip file when it is being requested by a user would alleviate this somewhat, as you only need storage space when the user places a request. You could for instance use a temporary file system for this. The problem with this approach is that the user is required to wait an additional amount of time for the zip file to be created first before it can be offered for download.
Generating a zip file on the fly has in this sense many benefits. First of all, because the zip file is generated on the fly, the download starts immediately and the user does not have to wait. Secondly, because the file is immediately sent to the user, it does not need to be saved on the disk. Most likely, the zip compression algorithm will be fast enough that no significant reduction in download speed is noted, although this depends of course on the technical specifications of your system. Indeed, the only major drawback of this method is that the processor has to work harder during the download and multiple simultaneous downloads could give very high load averages.
We assume for the sake of simplicity that you have a particular method in your controller which gives an array of the locations of all the files which need to be sent to the user. In the example below, we have a Folder Model with a hasMany association to a File class. We have chosen these names purely for this example, for a real application I would not recommend using a class named 'File' as there already exists such a class in the CakePHP library!!
public function zip($id = null) {
$this->layout = 'empty';
$this->Folder->recursive = 1;
$this->set('folder', $this->Folder->read(null, $id));
}
Note that we have set an empty file for the layout. (i.e. no layout)
Our view file looks like the following:
<?php
header('Content-Type: application/zip');
header('Content-disposition: attachment; filename="'.$folder['Folder']['name'].'.zip"');
header('Content-Transfer-Encoding: binary');
mb_http_output('pass');
ob_clean();
flush();
$files = '';
foreach($folder['File'] as $file) {
$files .= '"'.$folder['path'].'" ';
}
$fp = popen('zip -r - '.$files, 'r');
$bufsize = 8192;
$buff = '';
while( !feof($fp) ) {
$buff = fread($fp, $bufsize);
ob_flush();
echo $buff;
}
pclose($fp);
?>
Let us go through the code. The first three lines inform the browser that we are going to send a zip file with binary encoding. Moreover, we can already give the file a name which we have conveniently extracted from our Folder object. The mb_http_output('pass'); line ensures us that PHP is not going to mess with the encoding. ob_clean() and flush() flush the output and write buffers.
Next, we convert the array of file locations to a string. We use this string to tell the zip program (make sure you have it installed on your operating system) which files to compress. The output of the zip program is not a zip file, but a stream, which is conveniently piped to PHP using the popen function and referenced by a resource handler.
Now comes the interesting part, via this resource handler a chunk of data is read (as set in $bufsize, which is 8kb) to the read buffer, the read buffer is then flushed and the output is echoed. In other words, in segments of 8kb the download is being 'fed'. This is a continuous loop until an end-of-file (EOF) is encountered. After that, the pipe is closed. This way, we only need a very small amount of memory because we fill up a chunk of 8kb, parse it to the browser and then release the memory. If we would let the zip program stream its output completely to the memory and then echo this, we would most likely encounter a memory allocation error.
No post data.
No querystring data.
To view Cookies, add CookieComponent to Controller
Query | Affected | Num. rows | Took (ms) | Actions |
---|---|---|---|---|
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`posts` AS `Post` WHERE `Post`.`id` = 16 | 1 | 1 | 0 | |
SELECT `Post`.`active`, `Post`.`id` FROM `ivofilot_nl`.`posts` AS `Post` WHERE `Post`.`id` = 16 LIMIT 1 | 1 | 1 | 0 | |
SELECT `Comment`.`email`, `Comment`.`comment`, `Comment`.`id`, `Comment`.`post_id`, `Comment`.`created` FROM `ivofilot_nl`.`comments` AS `Comment` WHERE `Comment`.`post_id` = (16) | 0 | 0 | 0 | |
SELECT `Tag`.`id`, `Tag`.`name`, `Tag`.`icon`, `Tag`.`color`, `PostsTag`.`post_id`, `PostsTag`.`tag_id` FROM `ivofilot_nl`.`tags` AS `Tag` JOIN `ivofilot_nl`.`posts_tags` AS `PostsTag` ON (`PostsTag`.`post_id` = 16 AND `PostsTag`.`tag_id` = `Tag`.`id`) | 1 | 1 | 0 | |
UPDATE `posts` SET `watched`=`watched`+1 WHERE `id`='16' | 1 | 1 | 0 | |
SELECT `Comment`.`id`, `Comment`.`email`, `Comment`.`comment`, `Comment`.`post_id`, `Comment`.`parent_id`, `Comment`.`lft`, `Comment`.`rght`, `Comment`.`active`, `Comment`.`code`, `Comment`.`deleted`, `Comment`.`created`, `Comment`.`modified`, `Post`.`id`, `Post`.`title`, `Post`.`content`, `Post`.`watched`, `Post`.`active`, `Post`.`created`, `Post`.`modified`, `ParentComment`.`id`, `ParentComment`.`email`, `ParentComment`.`comment`, `ParentComment`.`post_id`, `ParentComment`.`parent_id`, `ParentComment`.`lft`, `ParentComment`.`rght`, `ParentComment`.`active`, `ParentComment`.`code`, `ParentComment`.`deleted`, `ParentComment`.`created`, `ParentComment`.`modified` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 16 AND `Comment`.`active` = '1' | 0 | 0 | 1 | maybe slow |
SELECT `Tag`.`id`, `Tag`.`name`, `Tag`.`icon`, `Tag`.`color` FROM `ivofilot_nl`.`tags` AS `Tag` inner JOIN `ivofilot_nl`.`posts_tags` AS `PostsTag` ON (`Tag`.`id` = `PostsTag`.`tag_id`) inner JOIN `ivofilot_nl`.`posts` AS `Post` ON (`PostsTag`.`post_id` = `Post`.`id`) WHERE `Post`.`id` = 16 | 1 | 1 | 0 | |
SELECT `Post`.`id`, `Post`.`title`, `Post`.`content`, `Post`.`watched`, `Post`.`active`, `Post`.`created`, `Post`.`modified`, `PostsTag`.`post_id`, `PostsTag`.`tag_id` FROM `ivofilot_nl`.`posts` AS `Post` JOIN `ivofilot_nl`.`posts_tags` AS `PostsTag` ON (`PostsTag`.`tag_id` = 6 AND `PostsTag`.`post_id` = `Post`.`id`) | 4 | 4 | 0 | |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 11 | 1 | 1 | 0 | |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 14 | 1 | 1 | 0 | |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 15 | 1 | 1 | 0 | |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 16 | 1 | 1 | 0 | |
SELECT `Post`.`id`, `Post`.`title`, `Post`.`content`, `Post`.`watched`, `Post`.`active`, `Post`.`created`, `Post`.`modified` FROM `ivofilot_nl`.`posts` AS `Post` WHERE `Post`.`id` = 16 LIMIT 1 | 1 | 1 | 0 | |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 16 | 1 | 1 | 0 |
Peak Memory Use 4.75 MB
Message | Memory use |
---|---|
Component initialization | 992 KB |
Controller action start | 1.02 MB |
Controller render start | 1.67 MB |
View render complete | 2.09 MB |
Total Request Time: 276 (ms)
Message | Time in ms | Graph |
---|---|---|
Core Processing (Derived from $_SERVER["REQUEST_TIME"]) | 63.87 | |
Event: Controller.initialize | 0.18 | |
Event: Controller.startup | 3.74 | |
Controller action | 106.77 | |
Event: Controller.beforeRender | 30.15 | |
» Processing toolbar data | 30.10 | |
Rendering View | 33.92 | |
» Event: View.beforeRender | 0.02 | |
» Rendering APP/View/Posts/view.ctp | 31.63 | |
» » Rendering APP/View/Elements/code_highlighting.ctp | 3.39 | |
» » Rendering APP/View/Elements/post.commentform.ctp | 18.13 | |
» » » Rendering APP/View/Elements/post.comment.captcha.ctp | 1.70 | |
» » Rendering APP/View/Elements/post.comments.ctp | 0.11 | |
» » Rendering APP/View/Elements/post.relatedpost.ctp | 0.25 | |
» Event: View.afterRender | 0.01 | |
» Event: View.beforeLayout | 0.01 | |
» Rendering APP/View/Layouts/default.ctp | 1.29 | |
» » Rendering APP/View/Elements/navbar.ctp | 0.27 | |
» » Rendering APP/View/Elements/footer.ctp | 0.22 | |
» » » Rendering APP/View/Elements/biography.ctp | 0.07 | |
Event: View.afterLayout | 0.00 |
Constant | Value |
---|---|
CONFIG | /customers/e/2/e/ivofilot.nl/httpd.www/app/Config/ |
Constant | Value |
---|---|
APP | /customers/e/2/e/ivofilot.nl/httpd.www/app/ |
APP_DIR | app |
APPLIBS | /customers/e/2/e/ivofilot.nl/httpd.www/app/Lib/ |
CACHE | /customers/e/2/e/ivofilot.nl/httpd.www/app/tmp/cache/ |
CAKE | /customers/e/2/e/ivofilot.nl/httpd.www/lib/Cake/ |
CAKE_CORE_INCLUDE_PATH | /customers/e/2/e/ivofilot.nl/httpd.www/lib |
CORE_PATH | /customers/e/2/e/ivofilot.nl/httpd.www/lib/ |
CAKE_VERSION | 2.10.13 |
CSS | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/css/ |
CSS_URL | css/ |
DS | / |
FULL_BASE_URL | https://ivofilot.nl |
IMAGES | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/img/ |
IMAGES_URL | img/ |
JS | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/js/ |
JS_URL | js/ |
LOGS | /customers/e/2/e/ivofilot.nl/httpd.www/app/tmp/logs/ |
ROOT | /customers/e/2/e/ivofilot.nl/httpd.www |
TESTS | /customers/e/2/e/ivofilot.nl/httpd.www/app/Test/ |
TMP | /customers/e/2/e/ivofilot.nl/httpd.www/app/tmp/ |
VENDORS | /customers/e/2/e/ivofilot.nl/httpd.www/vendors/ |
WEBROOT_DIR | webroot |
WWW_ROOT | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/ |
Environment Variable | Value |
---|---|
Php Version | 7.4.14 |
Onecom Domain Name | ivofilot.nl |
Onecom Domain Root | /customers/e/2/e/ivofilot.nl/ |
Onecom Memorylimit | 1073741824 |
Onecom Cpu Shares | 1024 |
Onecom Exec | latest |
Onecom Dir Layout Ver | 0 |
Content Length | 0 |
Http Connection | close |
Script Name | /app/webroot/index.php |
Request Uri | /posts/view/16/On+the+fly+generation+of+zip+files |
Query String | |
Request Method | GET |
Server Protocol | HTTP/1.1 |
Gateway Interface | CGI/1.1 |
Redirect Url | /app/webroot/posts/view/16/On+the+fly+generation+of+zip+files |
Remote Port | 41440 |
Script Filename | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/index.php |
Server Admin | support@one.com |
Context Document Root | /var/www |
Context Prefix | |
Request Scheme | https |
Remote Addr | 184.72.102.217 |
Server Port | 80 |
Server Addr | 10.27.35.20 |
Server Name | ivofilot.nl |
Server Software | Apache |
Server Signature | |
Path | /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin |
Http X Varnish | 971606230 |
Http Accept Encoding | gzip |
Http Host | ivofilot.nl |
Http X Onecom Host | ivofilot.nl |
Http X Forwarded Proto | https |
Http X Onecom Forwarded Proto | https |
Http X Forwarded For | 184.72.102.217 |
Http Accept Language | en-US,en;q=0.5 |
Http Accept | text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 |
Http User Agent | CCBot/2.0 (https://commoncrawl.org/faq/) |
Env Vcv Env Addons Id | one.com |
Env Vcv Token Url | https://wpapi.one.com/api/v1.0/plugins/visualcomposer/activate |
Onecom One Photo Url | https://onephoto.one.com/domain_discover |
Onecom Wp Addons Api | https://wpapi.one.com |
Onecom Webshop Host | webshop2.cst.webpod8-cph3.one.com |
Https | on |
Onecom Tmpdir | /customers/e/2/e/ivofilot.nl//tmp |
Domain Name | ivofilot.nl |
Onecom Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Redirect Status | 200 |
Redirect Env Vcv Env Addons Id | one.com |
Redirect Env Vcv Token Url | https://wpapi.one.com/api/v1.0/plugins/visualcomposer/activate |
Redirect Onecom One Photo Url | https://onephoto.one.com/domain_discover |
Redirect Onecom Wp Addons Api | https://wpapi.one.com |
Redirect Onecom Webshop Host | webshop2.cst.webpod8-cph3.one.com |
Redirect Https | on |
Redirect Onecom Cpu Shares | 1024 |
Redirect Onecom Memorylimit | 1073741824 |
Redirect Onecom Exec | latest |
Redirect Onecom Dir Layout Ver | 0 |
Redirect Onecom Tmpdir | /customers/e/2/e/ivofilot.nl//tmp |
Redirect Onecom Domain Root | /customers/e/2/e/ivofilot.nl/ |
Redirect Onecom Domain Name | ivofilot.nl |
Redirect Domain Name | ivofilot.nl |
Redirect Onecom Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Redirect Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Redirect Redirect Status | 200 |
Redirect Redirect Env Vcv Env Addons Id | one.com |
Redirect Redirect Env Vcv Token Url | https://wpapi.one.com/api/v1.0/plugins/visualcomposer/activate |
Redirect Redirect Onecom One Photo Url | https://onephoto.one.com/domain_discover |
Redirect Redirect Onecom Wp Addons Api | https://wpapi.one.com |
Redirect Redirect Onecom Webshop Host | webshop2.cst.webpod8-cph3.one.com |
Redirect Redirect Https | on |
Redirect Redirect Onecom Cpu Shares | 1024 |
Redirect Redirect Onecom Memorylimit | 1073741824 |
Redirect Redirect Onecom Exec | latest |
Redirect Redirect Onecom Dir Layout Ver | 0 |
Redirect Redirect Onecom Tmpdir | /customers/e/2/e/ivofilot.nl//tmp |
Redirect Redirect Onecom Domain Root | /customers/e/2/e/ivofilot.nl/ |
Redirect Redirect Onecom Domain Name | ivofilot.nl |
Redirect Redirect Domain Name | ivofilot.nl |
Redirect Redirect Onecom Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Redirect Redirect Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Fcgi Role | RESPONDER |
Php Self | /app/webroot/index.php |
Request Time Float | 1611319498.8131 |
Request Time | 1611319498 |