Support community for TTG plugins and products.
NOTICE
The Turning Gate's Community has moved to a new home, at https://discourse.theturninggate.net.
This forum is now closed, and exists here as a read-only archive.
You are not logged in.
I'm trying to figure out why Publish Now of large albums fails immediately, but smaller one, or many smaller ones concurrently all complete. Large album dies after updating none to a few images, while the smaller ones may each to 20 to a hundred and run for a long time without issue. On the large albums it only breaks after it is being published the first time when it is large, there after it breaks immediately. Error is Code 403 in a publisher pop up
PHP.INI limits?
Buffers
Objects
...
BackLight?
Photos per page (and what impact does this have in vegas?)
...
LR Classic?
Publisher handlers
...
Next up is probably a binary search to see if I can determine what the limit is.
Offline
Photos per page (and what impact does this have in vegas?)
I just tested this: it will limit the number of images in the slide show. But there is no pagination. So if you set it to 20, you'll have a slideshow of the first 20 images and that's it.
(I don't know the answers to your other questions)
Rod
Just a user with way too much time on his hands.
www.rodbarbee.com
ttg-tips.com, Backlight 2/3 test site
Offline
Have you tried publishing the 'large' album under another name? If it exhibits the same behavior (breaks after a while the first time then immediately after that) then my guess is your host thinks it is blocking an attack to the URL of the album. Maybe there's a way to whitelist that specific URL and see if that 'solves' the issue.
Charlie
www.stalkinglight.com
Offline
Yes, I have about 4 times.
I'm testing the size limit.
:-)
Offline
How big is your album? I have a couple that are around 250 images and they publish fine.
Charlie
www.stalkinglight.com
Offline
2350 was the first try.
Tried 550 and it fails
468 fails
Offline
I have a 300+ that works just fine.
Offline
That is a big album. The binary search you mentioned is probably your best bet.
Charlie
www.stalkinglight.com
Offline
You need to resolve your Wordfence issue.
For the last few days all I see when visiting your site is a long message starting with “Your access to this site has been limited”. Till then it is likely that Wordfence’s rate limiting is not enjoying having 3100 page calls within a matter of seconds.
What is the API URL you have pointed your Publisher instance to?
Offline
Offline
Well, it seems like it is a php memory limit.
Contacted Bluehost. They looked at the errors and said increase the php memory (which they did) and the problem boundary changed.
I'm not sure if I can make the php memory big enough. I currently have it at 1 GB.
Enhancement request would be to break up the script calls in to smaller chunks that the current all at once php call.
Offline
They looked at the errors
Which errors, in which file? Have you posted these errors in this forum? Please post them here.
I'm not sure if I can make the php memory big enough.
What makes you not sure? Is it working, or not? What have you been able to set it to?
Enhancement request would be to break up the script calls in to smaller chunks that the current all at once php call.
Calls are made for individual photos, hence repeatedly asking for you to resolve what appears to be a Wordfence block on multiple requests. Are there any particular requests that Bluehost has found have exceeded memory limits?
Here is what I continue to see when visiting http://patricklynchphotography.com/, including your Publisher API:
Offline
I didn't get that error visiting his site, maybe he has country blocking enabled?
Charlie
www.stalkinglight.com
Offline
I'm able to access his site as well.
Rod
Just a user with way too much time on his hands.
www.rodbarbee.com
ttg-tips.com, Backlight 2/3 test site
Offline
I'm getting this message in the browser when I visit your Galleries page:
No input file specified.
see if this helps: http://community.theturninggate.net/vie … hp?id=6287
Rod
Just a user with way too much time on his hands.
www.rodbarbee.com
ttg-tips.com, Backlight 2/3 test site
Offline
Ben,
I do have country blocking on; I am slightly paranoid. I work in Silicon valley in the high tech security sector. If you have a fixed IP or domain, I'll whitelist you.
I have been trying various php options to see what gave the best performance. There was a period where I tried php 7, and FastCGI that worked well for a couple of hours then died with the No Input specified error. so I switched it back to php 7 Single instance.
Hope that explains the errors folks were seeing.
What I found was the php memory limits for a script execution were set at 128K, and that was what was killing the publish script.
Changing it to 1000M (php.ini) did the trick (still took a while). Also since I have multiple sites, the single instance choice made administration much easier.
Thank you all who have kibitzed and helped to resolve the issue.
Offline
Thanks for clarifying. Are you sure it was only 128K and not 128M? I take it all the issues then are now resolved?
Offline
You are correct on both accounts
it was 128M
And the issue is resolved. I wonder how many of the other rendition failed with Code 403 errors are actually the same thing, needing more memory in php?
Yes, consider this resolved.
Offline
Can you find out which URL call caused the memory issue? If you look in ttg.log, and search for 403, the preceding API call log entry should show which page calls failed.
Which other Code 403 errors do you mean? There is only one other topic I could find with a 403 rendition error, and that was caused by a security setting.
(there are other posts regarding 403s, but they are not related to publishing)
The most common Publisher server error is caused by GoDaddy's mod_security rules, e.g. the cause of this error: http://community.theturninggate.net/vie … hp?id=8117
Offline
Ben, I will look and let you know.
A search of issues showed a variety of 403 errors. That is what I was thinking about.
Offline
Most common error was the line 605 (maybe 90%) Hope this helps.
[24-Oct-2017 02:54:31] TRACE: returning: {"status":"success","upload":{"filename":"_MG_2455.jpg","type":"application\/octet-stream","size":403843},"asset":{"album_id":30572,"is_complete":"no","filename":"_MG_2455.jpg","url":"http:\/\/www.patricklynchphotography.com\/galleries\/ZALL1\/","title":"_MG_2455.jpg","id":"30572_63157"},"photoId":"30572_63157"}
[24-Oct-2017 02:54:31] TRACE: returning: {"status":"error","message":"Unexpected error: Undefined variable: old_filename in PublisherDelegate.php on line 605"}
[24-Oct-2017 02:54:29] TRACE: returning: {"status":"success","upload":{"filename":"PJL20150422-Friday-Harbor-5140.jpg","type":"application\/octet-stream","size":45847},"asset":{"album_id":39727,"is_complete":"no","filename":"PJL20150422-Friday-Harbor-5140.jpg","url":"http:\/\/www.patricklynchphotography.com\/galleries\/All-other-groups\/","title":"PJL20150422-Friday-Harbor-5140.jpg","id":"39727_41555"},"photoId":"39727_41555"}
[24-Oct-2017 02:54:29] TRACE: returning: {"status":"error","message":"Unexpected error: Undefined variable: old_filename in PublisherDelegate.php on line 605"}
[24-Oct-2017 02:54:31] TRACE: NOT get_magic_quotes_gpc
[24-Oct-2017 02:54:31] TRACE: $json_parameters 2: {"action":"uploadRendition","fileSize":403843,"galleryName":"galleries","height":"960","id":30572,"isVirtualCopy":false,"name":"_MG_2455.jpg","photoId":"","renditionPath":"photos","template":"highslide_php_plugins_paged","version":"3.2.2","width":587}
[24-O
24-Oct-2017 02:54:35] TRACE: $destFilename: PJL20150422-Friday-Harbor-5154.jpg
[24-Oct-2017 02:54:35] TRACE: returning: {"status":"success","upload":{"filename":"PJL20150422-Friday-Harbor-5154.jpg","type":"application\/octet-stream","size":31273},"asset":{"album_id":39727,"is_complete":"no","filename":"PJL20150422-Friday-Harbor-5154.jpg","url":"http:\/\/www.patricklynchphotography.com\/galleries\/All-other-groups\/","title":"PJL20150422-Friday-Harbor-5154.jpg","id":"39727_39842"},"photoId":"39727_39842"}
[2
[24-Oct-2017 02:54:37] TRACE: $old_filename: _MG_2455.jpg
[24-Oct-2017 02:54:37] TRACE: $destFilename: _MG_2455.jpg
[24-Oct-2017 02:54:37] TRACE: returning: {"status":"success","upload":{"filename":"_MG_2455.jpg","type":"application\/octet-stream","size":71498},"asset":{"id":"30572_63157","album_id":30572,"filename":"_MG_2455.jpg","title":"_MG_2455.jpg","url":"http:\/\/www.patricklynchphotography.com\/galleries\/ZALL1\/","capture_time":null,"sequence":"0","latitude":null,"longitude":null,"is_complete":"no","created":"2017-10-24 02:54:31","modified":null},"photoId":"30572_63157"}
[24-Oct-2017 02:54:37] TRACE: returning: {"status":"error","message":"Unexpected error: Unterminated comment starting line 411 in rules.php on line 828"}
[24-Oct-2017 02:54:59] TRACE: $json_parameters 2: {"action":"uploadRendition","fileSize":25268,"galleryName":"galleries","height":167,"id":30572,"isVirtualCopy":false,"name":"_MG_2463.jpg","photoId":"30572_56323","renditionPath":"thumbnails","template":"highslide_php_plugins_paged","version":"3.2.2","width":"250"}
[24-Oct-2017 02:54:59] TRACE: API KEY: BackLight2016
[24-Oct-2017 02:54:59] TRACE: ========================================== [ uploadRendition ] ==========================================
[24-Oct-2017 02:54:59] TRACE: $parameters: Array( [action] => uploadRendition [fileSize] => 25268 [galleryName] => galleries [height] => 167 [id] => 30572 [isVirtualCopy] => [name] => _MG_2463.jpg [photoId] => 30572_56323 [renditionPath] => thumbnails [template] => highslide_php_plugins_paged [version] => 3.2.2 [width] => 250)
[24-Oct-2017 02:54:59] TRACE: text for hashing: BackLight2016{"action":"uploadRendition","fileSize":25268,"galleryName":"galleries","height":167,"id":30572,"isVirtualCopy":false,"name":"_MG_2463.jpg","photoId":"30572_56323","renditionPath":"thumbnails","template":"highslide_php_plugins_paged","version":"3.2.2","width":"250"}
[24-Oct-2017 02:54:59] TRACE: cs: 5175cc476c41f01bd0235cd95a37e4b1
[24-Oct-2017 02:54:59] TRACE: md5: 5175cc476c41f01bd0235cd95a37e4b1
[24-Oct-2017 02:54:59] TRACE: Authenticated
[24-Oct-2017 02:54:59] TRACE: path: /home1/sierraf8/public_html/patricklynchphotography/galleries||/ZALL1
[24-Oct-2017 02:54:59] TRACE: fileSize: 25268
[24-Oct-2017 02:54:59] TRACE: $old_filename: _MG_2463.jpg
[24-Oct-2017 02:54:59] TRACE: $destFilename: _MG_2463.jpg
[24-Oct-2017 02:54:59] TRACE: returning: {"status":"success","upload":{"filename":"_MG_2463.jpg","type":"application\/octet-stream","size":25268},"asset":{"id":"30572_56323","album_id":30572,"filename":"_MG_2463.jpg","title":"_MG_2463.jpg","url":"http:\/\/www.patricklynchphotography.com\/galleries\/ZALL1\/","capture_time":null,"sequence":"0","latitude":null,"longitude":null,"is_complete":"no","created":"2017-10-24 02:54:56","modified":null},"photoId":"30572_56323"}
[24-Oct-2017 02:54:59] TRACE: returning: {"status":"error","message":"Unexpected error: Unexpected character in input: '\u0003' (ASCII=3) state=0 in rules.php on line 828"}
[24-O
========================================== [ uploadRendition ] ==========================================
[24-Oct-2017 20:23:12] TRACE: $parameters: Array( [action] => uploadRendition [fileSize] => 214553 [galleryName] => galleries [height] => 640 [id] => 69175 [isVirtualCopy] => [name] => PL20130326-Mesa-Arch-5771_2.jpg [photoId] => 69175_84795 [renditionPath] => photos [template] => highslide_php_plugins_paged [version] => 3.2.2 [width] => 960)
[24-Oct-2017 20:23:12] TRACE: text for hashing: BackLight2016{"action":"uploadRendition","fileSize":214553,"galleryName":"galleries","height":640,"id":69175,"isVirtualCopy":false,"name":"PL20130326-Mesa-Arch-5771_2.jpg","photoId":"69175_84795","renditionPath":"photos","template":"highslide_php_plugins_paged","version":"3.2.2","width":"960"}
[24-Oct-2017 20:23:12] TRACE: cs: da1218e5614c0702f524a166af4a2b7a
[24-Oct-2017 20:23:12] TRACE: md5: da1218e5614c0702f524a166af4a2b7a
[24-Oct-2017 20:23:12] TRACE: Authenticated
[24-Oct-2017 20:23:12] TRACE: loaded 13 album sets
[24-Oct-2017 20:23:12] TRACE: path: /home1/sierraf8/public_html/patricklynchphotography/galleries||/00landscapes/southwest/arches-canyonlands
[24-Oct-2017 20:23:12] TRACE: fileSize: 214553
[24-Oct-2017 20:23:12] TRACE: $old_filename: PL20130326-Mesa-Arch-5771_2.jpg
[24-Oct-2017 20:23:12] TRACE: $destFilename: PL20130326-Mesa-Arch-5771_2.jpg
[24-Oct-2017 20:23:12] TRACE: loaded 13 album sets
[24-Oct-2017 20:23:12] TRACE: returning: {"status":"success","upload":{"filename":"PL20130326-Mesa-Arch-5771_2.jpg","type":"application\/octet-stream","size":214553},"asset":{"id":"69175_84795","album_id":69175,"filename":"PL20130326-Mesa-Arch-5771_2.jpg","title":"PL20130326-Mesa-Arch-5771_2.jpg","url":"http:\/\/www.patricklynchphotography.com\/galleries\/00landscapes\/southwest\/arches-canyonlands\/","capture_time":"2013-03-26 06:54:32","sequence":"0","latitude":"38.387965","longitude":"-109.86360333333","is_complete":"no","created":"2014-07-24 02:59:06","modified":"2017-10-20 14:23:35"},"photoId":"69175_84795"}
[24-Oct-2017 20:23:12] TRACE: returning: {"status":"error","message":"Unexpected error: Unexpected character in input: '\u0015' (ASCII=21) state=0 in rules.php on line 856"}
[24-Oct-2017 20:23:14] TRACE: NOT get_magic_quotes_gpc
[24-Oct-2017 20:23:14] TRACE: $json_parameters 2: {"action":"uploadRendition","fileSize":39184,"galleryName":"galleries","height":167,"id":69175,"isVirtualCopy":false,"name":"PL20130326-Mesa-Arch-5771_2.jpg","photoId":"69175_84795","renditionPath":"thumbnails","template":"highslide_php_plugins_paged","version":"3.2.2","width":"250"}
[24-Oct-2017 20:23:14] TRACE: API KEY: BackLight2016
[24-Oct-2017 20:23:14] TRACE: ========================================== [ uploadRendition ] ==========================================
[24-Oct-2017 20:23:14] TRACE: $parameters: Array( [action] => uploadRendition [fileSize] => 39184 [galleryName] => galleries [height] => 167 [id] => 69175 [isVirtualCopy] => [name] => PL20130326-Mesa-Arch-5771_2.jpg [photoId] => 69175_84795 [renditionPath] => thumbnails [template] => highslide_php_plugins_paged [version] => 3.2.2 [width] => 250)
[24-Oct-2017 20:23:14] TRACE: text for hashing: BackLight2016{"action":"uploadRendition","fileSize":39184,"galleryName":"galleries","height":167,"id":69175,"isVirtualCopy":false,"name":"PL20130326-Mesa-Arch-5771_2.jpg","photoId":"69175_84795","renditionPath":"thumbnails","template":"highslide_php_plugins_paged","version":"3.2.2","width":"250"}
[24-Oct-2017 20:23:14] TRACE: cs: ba3f21b0458ca1ad9f6fa4d26a39fb5a
[24-Oct-2017 20:23:14] TRACE: md5: ba3f21b0458ca1ad9f6fa4d26a39fb5a
[24-Oct-2017 20:23:14] TRACE: Authenticated
[24-Oct-2017 20:23:14] TRACE: loaded 13 album sets
[24-Oct-2017 20:23:14] TRACE: path: /home1/sierraf8/pu
Offline
Wrong logs, here are errors from ttg.log
10/20/2017 06:38:02 DEBUG callRemoteAPI: http://www.PatrickLynchPhotography.com/backlight/publisher/?source=lightroom?parameters={"action":"saveMetadata","galleryName":"galleries","id":25753,"metadata_name_1":"ttg_latitude","metadata_name_10":"latitude","metadata_name_11":"photo_title","metadata_name_12":"ttg_longitude","metadata_name_2":"ttg_keywords","metadata_name_3":"img_label","metadata_name_4":"ttg_capture_time","metadata_name_5":"caption","metadata_name_6":"photo_caption","metadata_name_7":"color-label","metadata_name_8":"title","metadata_name_9":"longitude","metadata_value_1":"36.028021666667","metadata_value_10":"36.028021666667","metadata_value_11":"Big Sur poppies","metadata_value_12":"-121.57363166667","metadata_value_2":"Big Sur coast, Poppies, Wildflowers, flowers","metadata_value_3":"Big Sur poppies","metadata_value_4":"2012-05-05 13:15:46","metadata_value_5":"Roadside field","metadata_value_6":"Roadside field Copyright 2011 by Patrick Lynch ","metadata_value_7":"","metadata_value_8":"Big Sur poppies","metadata_value_9":"-121.57363166667","photoId":"25753_85608","version":"3.2.2"}&cs=91a272daba901631eb5546dcdd1fb701
10/20/2017 10:33:54 DEBUG UploadRendition: url=http://www.PatrickLynchPhotography.com/backlight/publisher/?source=lightroom; parameters={"action":"uploadRendition","fileSize":289365,"galleryName":"galleries","height":640,"id":37333,"isVirtualCopy":false,"name":"PL20140414-Escalante-0840.jpg","photoId":"","renditionPath":"photos","template":"highslide_php_plugins_paged","version":"3.2.2","width":"960"}
10/20/2017 11:05:17 DEBUG UploadRendition: url=http://www.PatrickLynchPhotography.com/backlight/publisher/?source=lightroom; parameters={"action":"uploadRendition","fileSize":289365,"galleryName":"galleries","height":640,"id":37333,"isVirtualCopy":false,"name":"PL20140414-Escalante-0840.jpg","photoId":"","renditionPath":"photos","template":"highslide_php_plugins_paged","version":"3.2.2","width":"960"}
10/20/2017 13:10:00 DEBUG sharpening path: photos
url=http://www.PatrickLynchPhotography.com/backlight/publisher/?source=lightroom; parameters={"action":"uploadRendition","fileSize":289365,"galleryName":"galleries","height":640,"id":37333,"isVirtualCopy":false,"name":"PL20140414-Escalante-0840.jpg","photoId":"","renditionPath":"photos","template":"highslide_php_plugins_paged","version":"3.2.2","width":"960"}
Offline
Thanks for the logs. There are two main types of errors that I can see:
1. Undefined variable $old_filename. Which version of Publisher are you using? (this can be found by logging into Backlight and clicking Backlight > Publisher in the menu).
2. Errors on rules.php. This doesn't look to be Backlight-related. Can you find the file rules.php on your server (perhaps somewhere under wordpress or the wordfence module?) and copy here the two lines, 828 and 856?
None of the errors seem memory-related.
Offline
Backlight Publisher version: 3.2.2
still searching
Offline
/public_html/patricklynchphotography/blog/wp-content/plugins/wordfence/vendor/wordfence/wf-waf/src/lib/rules.php
828: $tokens = @token_get_all($data);
856: @token_get_all($testBytes);
Whole function is:
public function fileHasPHP($subject) {
$request = $this->getWAF()->getRequest();
$files = $request->getFiles();
if (!is_array($files)) {
return false;
}
foreach ($files as $file) {
if ($file['name'] == (string) $subject) {
$fh = @fopen($file['tmp_name'], 'r');
if (!$fh) {
continue;
}
$totalRead = 0;
$insideOpenTag = false;
$hasExecutablePHP = false;
$possiblyHasExecutablePHP = false;
$hasOpenParen = false;
$hasCloseParen = false;
$backtickCount = 0;
$wrappedTokenCheckBytes = '';
$maxTokenSize = 15; //__halt_compiler
$possibleWrappedTokens = array('<?php', '<?=', '<?', '?>', 'exit', 'new', 'clone', 'echo', 'print', 'require', 'include', 'require_once', 'include_once', '__halt_compiler');
$readsize = 100 * 1024; //100k at a time
while (!feof($fh)) {
$data = fread($fh, $readsize);
$actualReadsize = strlen($data);
$totalRead += $actualReadsize;
if ($totalRead < 1) {
break;
}
//Make sure we didn't miss PHP split over a chunking boundary
$wrappedCheckLength = strlen($wrappedTokenCheckBytes);
if ($wrappedCheckLength > 0) {
$testBytes = $wrappedTokenCheckBytes . substr($data, 0, min($maxTokenSize, $actualReadsize));
foreach ($possibleWrappedTokens as $t) {
$position = strpos($testBytes, $t);
if ($position !== false && $position < $wrappedCheckLength && $position + strlen($t) >= $wrappedCheckLength) { //Found a token that starts before this segment of data and ends within it
$data = substr($wrappedTokenCheckBytes, $position) . $data;
break;
}
}
}
//Make sure it tokenizes correctly if chunked
if ($insideOpenTag) {
if ($possiblyHasExecutablePHP) {
$data = '<?= ' . $data;
}
else {
$data = '<?php ' . $data;
}
}
//Tokenize the data and check for PHP
$this->_resetErrors();
$tokens = @token_get_all($data);
$error = error_get_last();
if ($error !== null && stripos($error['message'], 'Unexpected character in input') !== false) {
break;
}
if ($error !== null && feof($fh) && stripos($error['message'], 'Unterminated comment') !== false) {
break;
}
$offset = 0;
foreach ($tokens as $token) {
if (is_array($token)) {
$offset += strlen($token[1]);
switch ($token[0]) {
case T_OPEN_TAG:
$insideOpenTag = true;
$hasOpenParen = false;
$hasCloseParen = false;
$backtickCount = 0;
$possiblyHasExecutablePHP = false;
if ($error !== null && stripos($error['message'], 'Unterminated comment') !== false) {
$testOffset = $offset - strlen($token[1]);
$commentStart = strpos($data, '/*', $testOffset);
if ($commentStart !== false) {
$testBytes = substr($data, $testOffset, $commentStart - $testOffset);
$this->_resetErrors();
@token_get_all($testBytes);
$error = error_get_last();
if ($error !== null && stripos($error['message'], 'Unexpected character in input') !== false) {
break 3;
}
}
}
break;
case T_OPEN_TAG_WITH_ECHO:
$insideOpenTag = true;
$hasOpenParen = false;
$hasCloseParen = false;
$backtickCount = 0;
$possiblyHasExecutablePHP = true;
if ($error !== null && stripos($error['message'], 'Unterminated comment') !== false) {
$testOffset = $offset - strlen($token[1]);
$commentStart = strpos($data, '/*', $testOffset);
if ($commentStart !== false) {
$testBytes = substr($data, $testOffset, $commentStart - $testOffset);
$this->_resetErrors();
@token_get_all($testBytes);
$error = error_get_last();
if ($error !== null && stripos($error['message'], 'Unexpected character in input') !== false) {
break 3;
}
}
}
break;
case T_CLOSE_TAG:
$insideOpenTag = false;
if ($possiblyHasExecutablePHP) {
$hasExecutablePHP = true; //Assume the echo short tag outputted something useful
}
break 2;
case T_NEW:
case T_CLONE:
case T_ECHO:
case T_PRINT:
case T_REQUIRE:
case T_INCLUDE:
case T_REQUIRE_ONCE:
case T_INCLUDE_ONCE:
case T_HALT_COMPILER:
case T_EXIT:
$hasExecutablePHP = true;
break 2;
}
}
else {
$offset += strlen($token);
switch ($token) {
case '(':
$hasOpenParen = true;
break;
case ')':
$hasCloseParen = true;
break;
case '`':
$backtickCount++;
break;
}
}
if (!$hasExecutablePHP && (($hasOpenParen && $hasCloseParen) || ($backtickCount > 1 && $backtickCount % 2 === 0))) {
$hasExecutablePHP = true;
break;
}
}
if ($hasExecutablePHP) {
fclose($fh);
return true;
}
$wrappedTokenCheckBytes = substr($data, - min($maxTokenSize, $actualReadsize));
}
fclose($fh);
}
}
return false;
}
Offline