Planet-PHP
Offline Processing in PHP with Advanced Queuing
Offloading slow batch tasks to an external process is a common method of improving website responsiveness. One great way to initiate such background tasks in PHP is to use Oracle Streams Advanced Queuing in a producer-consumer message passing fashion. Oracle AQ is highly configurable. Messages can queued by multiple producers. Different consumers can filter messages. From PHP, the PL/SQL interface to AQ is used. There are also Java, C and HTTPS interfaces, allowing wide architectural freedom.
The following example simulates an application user registration system where the PHP application queues each new user's street address. An external system monitoring the queue can then fetch and process that address. In real life the external system might initiate a snail-mail welcome letter, or do further, slower automated validation on the address.
The following SQL*Plus script qcreate.sql creates a new Oracle user demoqueue with permission to create and use queues. A payload type for the address is created and a queue is set up for this payload.
-- qcreate.sql connect / as sysdba drop user demoqueue cascade; create user demoqueue identified by welcome; grant connect, resource to demoqueue; grant aq_administrator_role, aq_user_role to demoqueue; grant execute on dbms_aq to demoqueue; grant create type to demoqueue; connect demoqueue/welcome@localhost/orcl -- The data we want to queue create or replace type user_address_type as object ( name varchar2(10), address varchar2(50) ); / -- Create and start the queue begin dbms_aqadm.create_queue_table( queue_table => 'demoqueue.addr_queue_tab', queue_payload_type => 'demoqueue.user_address_type'); end; / begin dbms_aqadm.create_queue( queue_name => 'demoqueue.addr_queue', queue_table => 'demoqueue.addr_queue_tab'); end; / begin dbms_aqadm.start_queue( queue_name => 'demoqueue.addr_queue', enqueue => true); end; /
The script qhelper.sql creates two useful helper functions to enqueue and dequeue messages:
-- qhelper.sql
-- Helpful address enqueue/dequeue procedures
connect demoqueue/welcome@localhost/orcl
-- Put an address in the queue
create or replace procedure my_enq(name_p in varchar2, address_p in varchar2) as
user_address user_address_type;
enqueue_options dbms_aq.enqueue_options_t;
message_properties dbms_aq.message_properties_t;
enq_id raw(16);
begin
user_address := user_address_type(name_p, address_p);
dbms_aq.enqueue(queue_name => 'demoqueue.addr_queue',
enqueue_options => enqueue_options,
message_properties => message_properties,
payload => user_address,
msgid => enq_id);
commit;
end;
/
show errors
-- Get an address from the queue
create or replace procedure my_deq(name_p out varchar2, address_p out varchar2) as
dequeue_options dbms_aq.dequeue_options_t;
message_properties dbms_aq.message_properties_t;
user_address user_address_type;
enq_id raw(16);
begin
dbms_aq.dequeue(queue_name => 'demoqueue.addr_queue',
dequeue_options => dequeue_options,
message_properties => message_properties,
payload => user_address,
msgid => enq_id);
name_p := user_address.name;
address_p := user_address.address;
commit;
end;
/
show errors
The script newuser.php is the part of the PHP application that handles site registration for a new user. It queues a message containing their address and continues executing:
<?php
// newuser.php
$c = oci_connect("demoqueue", "welcome", "localhost/orcl");
// The new user details
$username = 'Fred';
$address = '500 Oracle Parkway';
// Enqueue the address for later offline handling
$s = oci_parse($c, "begin my_enq(:username, :address); end;");
oci_bind_by_name($s, ":username", $username, 10);
oci_bind_by_name($s, ":address", $address, 50);
$r = oci_execute($s);
// Continue executing
echo "Welcome $username\n";
?>
It executes an anonymous PL/SQL block to create and enqueue the address message. The immediate script output is simply the echoed welcome message:
Welcome Fred
Once this PHP script is executed, any application can dequeue the new message at its leisure. For example, the following SQL*Plus commands call the helper my_deq() dequeue function and displays the user details:
-- getuser.sql
connect demoqueue/welcome@localhost/orcl
set serveroutput on
declare
name varchar2(10);
address varchar2(50);
begin
my_deq(name, address);
dbms_output.put_line('Name : ' || name);
dbms_output.put_line('Address : ' || address);
end;
/
The
Truncated by Planet PHP, read more at the original (another 1511 bytes)
Slides: Behat & Beautiful APIs
Publishing Security Disclosures In Consumable Formats For Simpler Aggregation and Security Checking
This is a branch off from a separate discussion on the PHP-FIG mailing list about other ways the Framework Interoperability Group can encourage and foster wider interoperability among its member projects (and by extension, the whole PHP community). I’ll start by noting two interesting developments in recent months and one long standing best practice.
1. Launch of the SensioLabs Security Advisory CheckerThe SensioLabs Security Advisor Checker is described on its website as follows.
You manage your PHP project dependencies with Composer, right? But are you sure that your project does not depend on a package with known security issues? The SensioLabs security advisories checker is a simple tool, available as a web service or as an online application, that uses the information from your composer.lock file to check for known security vulnerabilities. This checker is a frontend for the security advisories database.
The service operates by having people submit vulnerability data, as YAML files, to a centralised Github repository through pull requests. The upside is that the vulnerability data can be peer reviewed and centrally dispersed either online or via a service API. The downside is that you need to find vulnerability disclosures and people to submit them. The service currently covers Symfony, Zend Framework, Doctrine, Twig and FriendsOfSymfony bundles. It’s a tiny sample of packages available through Composer. I’m also not entirely sure if it’s sufficiently fine grained to report vulnerabilities on a project’s sub-packages where you have no direct dependency on the aggregate package (e.g. using zendframework/zend-db instead of zendframework/zendframework). That said, this is a working model of a service for checking your dependencies.
That said, the service exhibits an ambitious idea – projects sharing their vulnerability disclosures or advisories in a way that allows for automatically checking if any of your projects need to have their dependencies updated for security reasons.
2. OWASP‘s Top 10 security risks for 2013 includes “A9 – Using Components with Known Vulnerabilities”This is a new entry onto OWASP’s Top 10 (which is currently at release candidate status for 2013). In summary, it recognises that applications are becoming ever more dependent on code not developed internally. We’ve had web application frameworks for years. Composer and Github have unleashed a storm of accessible libraries, bundles, modules, and other units of reuse that have revealed Not Invented Here (NIH Syndrome) as a psychological problem in ways not previously possible.
As reliance on externally controlled dependencies increases, so too does the risk of your applications using insecure dependencies. This is a risk that requires a lot of work to mitigate. For each dependency, you need to do a security review (no, I’m not kidding), check for security disclosures (whether voluntary or involuntary) and ensure that you end up rolling out to production with safe versions.
Quoting from the OWASP advice on preventing the use of components with known vulnerabilities…
One option is not to use compone
Truncated by Planet PHP, read more at the original (another 4681 bytes)
How we organize our websites
We recently migrated Where’s it Up to our fancy new hardware, it took a bit more effort than planned (some pains surrounding our use of MongoDB) but I’m incredibly happy with how things have ended up. As mentioned earlier we’ve purchased our own hardware, and have racked it with Peer 1 here in Toronto. We’ve installed a hypervisor, and are running different VMs for critical services: MySQL, Mongo, Web Production, Web Development, etc.
Our websites sit under /var/www, so Where’s it Up resides at /var/www/wheresitup.com/. Under that directory we have /noweb/apache/ which contains both wheresitup.com and dev.wheresitup.com, configuration files for apache. The entire /var/www/wheresitup.com directory tree resides nicely in our version control system. We hand off key configuration options to our websites through the use of Apache’s SetEnv, things like SetEnv mysql_host dev.mysql, these apache configuration options represent the only difference between the two code bases.
I’ve written or maintained code that implied the state (Dev/Production/Stage) based on the Host, directory, or other factors in the past. I much prefer grabbing an explicit constant. It feels cleaner, I don’t have to read up on which variables could have been manipulated by an attacker, and I can ask the exact question I want answered: Is this dev, rather than “is the url the one that means this is dev”.
This allows us to match our Development and Production virtual machines very closely, the only difference between the two is which apache configuration file is sym-linked under /etc/apache2/conf/sites-enabled. Clearly WebDev links to the dev.wheresitup.com file, and WebProd links to wheresitup.com. We actually cloned one machine to produce the other.
Keeping the configuration files so close also makes a lot of sense to me. If I’m adding a new constant on Dev, the immediate presence of Prod reminds me that I’ll need to add it there as well. Storing the entire site: PHP code, supporting apache configuration, etc, all in once place makes it easy to avoid forgetting anything (which is easy when it's a different file on a different server). The only exception is SSL certificates. We currently host a number of our projects with GitHub, and trust them as we might, we’re not willing to hand those to anyone else.
Dealing with duplicated code
Getting Started with PHP Zend Framework 2 for Oracle DB
This post shows the changes to the ZF2 tutorial application to allow it to run with Oracle Database 11gR2.
Oracle Database SQL identifiers are case insensitive by default so "select Abc from Xyz" is the same as "select abc from xyz". However the identifier metadata returned to programs like PHP is standardized to uppercase by default. After executing either query PHP knows that column "ABC" was selected from table "XYZ".
In PHP code, array indices and object attributes need to match the schema identifier case that is returned by the database. This is either done by using uppercase indices and attributes in the PHP code, or by forcing the SQL schema to case-sensitively use lower-case names.
The former approach is more common, and is shown here.
The instructions for creating the sample ZF2 application are here. Follow those steps as written, making the substitutions shown below.
SchemaIn Oracle 11gR2, the schema can be created like:
DROP USER ZF2 CASCADE;
CREATE USER ZF2 IDENTIFIED BY WELCOME
DEFAULT TABLESPACE USERS QUOTA UNLIMITED ON USERS
TEMPORARY TABLESPACE TEMP;
GRANT CREATE SESSION
, CREATE TABLE
, CREATE PROCEDURE
, CREATE SEQUENCE
, CREATE TRIGGER
, CREATE VIEW
, CREATE SYNONYM
, ALTER SESSION
TO ZF2;
CONNECT ZF2/WELCOME
CREATE TABLE ALBUM (
ID NUMBER NOT NULL,
ARTIST VARCHAR2(100) NOT NULL,
TITLE VARCHAR2(100) NOT NULL,
PRIMARY KEY (ID)
);
CREATE SEQUENCE ALBUMSEQ;
CREATE TRIGGER ALBUMTRIGGER BEFORE INSERT ON ALBUM FOR EACH ROW
BEGIN
:NEW.ID := ALBUMSEQ.NEXTVAL;
END;
/
INSERT INTO ALBUM (ARTIST, TITLE)
VALUES ('The Military Wives', 'In My Dreams');
INSERT INTO ALBUM (ARTIST, TITLE)
VALUES ('Adele', '21');
INSERT INTO ALBUM (ARTIST, TITLE)
VALUES ('Bruce Springsteen', 'Wrecking Ball (Deluxe)');
INSERT INTO ALBUM (ARTIST, TITLE)
VALUES ('Lana Del Rey', 'Born To Die');
INSERT INTO ALBUM (ARTIST, TITLE)
VALUES ('Gotye', 'Making Mirrors');
COMMIT;
Driver and Credentials
The driver and credentials are Oracle-specific. Always use the OCI8 adapter in ZF, since it is more stable and has better scalability. Specifying a character set will make connection faster.
zf2-tutorial/config/autoload/global.php: return array(
'db' => array(
- 'driver' => 'Pdo',
- 'dsn' => 'mysql:dbname=zf2tutorial;host=localhost',
- 'driver_options' => array(
- PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
- ),
+ 'driver' => 'OCI8',
+ 'connection_string' => 'localhost/orcl',
+ 'character_set' => 'AL32UTF8',
),
'service_manager' => array(
'factories' => array(
zf2-tutorial/config/autoload/local.php:
return array(
'db' => array(
- 'username' => 'YOUR USERNAME HERE',
- 'password' => 'YOUR USERNAME HERE',
+ 'username' => 'ZF2',
+ 'password' => 'WELCOME',
),
// Whether or not to enable a configuration cache.
// If enabled, the merged configuration will be cached and used in
Attribute & Index Changes
The rest of the application changes are just to handle the case of the Oracle identifiers correctly.
zf2-tutorial/module/Album/Module.php
$dbAdapter = $sm->get('Zend\Db\Adapter\Adapter');
$resultSetPrototype = new ResultSet();
$resultSetPrototype->setArrayObjectPrototype(new Album());
- return new TableGateway('album', $dbAdapter, null, $resultSetPrototype);
+ return new TableGateway('ALBUM', $dbAdapter, null, $resultSetPrototype);
},
),
);
zf2-tutorial/module/Album/view/album/album/add.phtml
$form->prepare();
echo $this->form()->openTag($form);
-echo $this->formHidden($form->get('id'));
-echo $this->formRow($form->get('title'));
-echo $this->formRow($form->get('artist'));
+echo $this->formHidden($form->get('ID'));
+echo $this->formRow($form->get('TITLE'));
+echo $this->formRow($form->get('ARTIST'));
echo $this->formSubmit($form->get('submit'));
echo $this->form()->closeTag();
zf2-tutorial/module/Album/view/album/album/delete.phtml
<h1><?php echo $this->escapeHtml($title); ?></h1> <p>Are you sure that you want to delete -'<?php echo $this->escapeHtml($album->title); ?>' by -'<?php echo $this->escapeHtml($album->artist); ?>'? +'<?php echo $this->escapeHtml($album->TITLE); ?>' by +'<?php echo $this->esc
Truncated by Planet PHP, read more at the original (another 7231 bytes)
Improve PHP session cookie security
Compiling PHP 5.5 From Scratch
Escaping in iCalendar and vCard
The #1 bug report in my vObject library (a library to parse and create iCalendar and vCard objects in PHP) is that it does a bad job escaping/un-escaping of values.
In particular, it double-escapes certain values, changing things like ; into
\\; and in other cases it's a bit too liberal un-escaping.
It's gotten to a point where I got so frustrated about this bug, I've been working all week on a new version of the parser.
Determined to do things right this time, I wanted to make sure I complied with all the relevant standards, in particular:
When I first wrote the vObject I naively thought that these formats were more or less the same. On the surface it does indeed seem that way, everything does seem to follow this basic structure:
BEGIN:VCARD
VERSION:4.0
FN:Evert Pot
END:VCARD
The nuances and slight difference between the specifications are enough to drive a simple person to madness though.
Just on the topic of ecaping values (the part after the :) the
specifications have the following to say:
vCard 2.1, as well as the other specs have a concept of 'compound' or multi-value properties. An example:
BEGIN:VCARD
VERSION:2.1
N:Pot;Evert;Middle;Dr.;M.D.
END:VCARD
As you can see, the N property has multiple values. Any of these values
may also contain a ;, which must be escaped as \;. So we also cannot
blindly encode a string and automatically add backslashes to any ; we see.
The semi-colons should only be escaped in the ADR, ORG and N fields,
but we can assume that backslashed semi-colons may also appear in other values.
Any property may have a parameter, a parameter looks a bit like this:
BEGIN:VCARD
VERSION:2.1
NOTE;ENCODING=QUOTED-PRINTABLE:Handsome guy, for sure..
END:VCARD
A parameter in vCard starts with a ;, has a name and a value. Only the colon
may be escaped in parameters, using \:.
If you somehow wanted to encode a real backslash though, there's no mention of escaping it as a double-backslash.
If you need newlines in any values, quoted-printable encoding must be used.
Other specs all encode newlines as \n or \N.
rfc2425 says that backslashes (\\), newlines (\N or \n) and comma's (\,)
must always be escaped, no exceptions.. Well except when the comma is used as
a delimiter for multiple values.
rfc2426 add semi-colon (\;) to this list, except when it's used as a
delimiter. Semi-colon is used as a delimiter in the N, ADR, GEO and
ORG fields. NICKNAME and CATEGORIES use comma's.
vCard also says that individual parts of ADR, and N may also contain
multiple values themselves, which are themselves split by a comma.
Quoted-printable is now deprecated, and should no longer be used.
Parameters have also changed. The new rule is that parameters must not
contain ;, : or ", unless they are surrounded by double-quotes, in which
case only " may not appear. Escaping of the colon character (\:) has
disappeared.
vCard 4 changes the interpretation of 3.0 a bit, and now states that semi-colons may be escaped, depending on the property.
The implication is that we need to maintain lists of properties, if they
support multiple- or compound-values and which delimiter they use
(, or ;).
Semi-colons are now used by N, ADR, ORG and CLIENTPIDMAP. Comma's are
used by NICKNAME, RELATED, CATEGORIES and PID.
Even though the spec does say that comma's must always be escaped, it does
appear to violate this rule in it's own examples, specifically the example
for GEO (which is no longer a compound float value, but a url).
iCalendar 2
Truncated by Planet PHP, read more at the original (another 1860 bytes)
Making better object oriented design decisions
Aura: New Site, New Logo, New Releases
Better Documentation for PHP internals - Lately in PHP podcast episode 35
That is one of the topics discussed by Manuel Lemos and Ernani Joppert in the episode 35 of the Lately In PHP podcast.
They also talked about having more optimized PHP opcodes, some interesting PHP feature proposals that got rejected, as well the article about the top version control systems used by PHP developers.
Listen to this podcast, or watch the hangout video, or read the transcript to learn more about this and other interesting PHP discussions.
Seriously: PHP 5.4.15 and PHP 5.3.25 really were released!
PHP 5.5.0RC1 is available
PHP 5.4.15 and PHP 5.3.25 released!
Using Grep to Find Security Vulnerabilities in PHP code
Read this article to learn how you can find certain types of security vulnerabilities very quickly using the grep program.
Setting Multiple Headers in a PHP Stream Context
Last week I tried to create a PHP stream context which set multiple headers; an Authorization header and a Content-Type header. All the examples I could find showed headers built up as a string with newlines added manually, which seemed pretty clunky and not-streams-like to me.
In fact, you've been able to pass this as an array since PHP 5.2.10, so to set multiple headers in the stream context, I just used this:
<?php $options = ["http" => [ "method" => "POST", "header" => ["Authorization: token " . $access_token, "Content-Type: application/json"], "content" => $data ]]; $context = stream_context_create($options);
The $access_token had been set elsewhere (in fact I usually put credentials in a separate file and exclude it from source control in an effort not to spread my access credentials further than I mean to!), and $data is already encoded as JSON. For completeness, you can make the POST request like this:
<?php // make the request $response = file_get_contents($url, false, $context);
Hopefully this will help someone else doing the same thing next time (or at least I know I can come back here when I can't remember!), the array approach seems more elegant and maintainable to me.
Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane
PHPUCEU
So this weekend I visited my hometown Berlin for the PHPUCEU. Actually this was in very close proximity to where I grew up, which was nice as I was staying at my parents place. Usually staying there requires a one hour ride to the hipster areas of Berlin to attend a conference. In this case it was just a 2 stop U-Bahn ride. That being said, this wasn't a "normal" conference. This was an unconference. As such attendees proposed talks they could give, but also talks that they would want to hear. Every morning every attendee would then have two votes for talks. The top voted talks would then be distributed across the 4 slots in the 3 available tracks. As such several of the sessions ended up being quite ad hoc with multiple people chipping in with what they new about the topic. What is also special about this event is that the sessions are just as long as the breaks to facilitate idea exchange about the session topics, but also about other topics. Overall I found this to be an absolutely thrilling experience.
Day one PHPCRI proposed a talk on PHPCR which was selected for the first session. I essentially gave the talk I had given multiple times before. But its still exciting to share the content repository vision for PHP. Looks like there might even be PHPCR support in the future for Jimdo and Magento?
Clean puppetI attended this session by Soenke, Jakon und Hans-Christian mainly because I am still quite a noob when it comes to Puppet and Chef. I must admit, the possibilities excite me but I still prefer others to do the development for me. Somehow I just don't like the DSL of Puppet nor the Rubyness of Chef. This session didn't change this but did highlight some important aspects, specifically that you would keep the concept of environments as for up the chain as possible.
Behavior Driven DevelopmentKore and Gaylord gave this session. Gaylord was representing the more business level perspective while Kore the more technical one. The exciting part about BDD is that BDD can really bring those two sides closer together. However the most important take away for me from this session is that Behat can now generate stubs for Context definitions, which will drastically reduce the effort for expanding the DSL for non developers. Gaylord mentioned that he thinks that using BDD seriously increases development time. Yet the benefits of additional testing and always uptodate documentation outweigh these costs. We have played with BDD at Liip several times, but its time we get more serious about it.
Evening programmI missed the 4th session of the day because I was chatting with people outside. After the last session ended we got together and each session was briefly summarized. We headed to a local beer garden which unfortunately was a bit overwhelmed with us and so the service was quite non existant. However we still stayed several hours. Talking about technology and organization challenges.
Day Two Agile documentationThis was the first session of the day for me. Soenke was making some important points about how documentation does not only provide benefit but also cost. And this cost is best managed by single sourcing documentation (ie. not duplicating documentation) and by making documentation executable (via BDD).
Hood.ieThis session was slightly sabotaged by github. Kore and Hans-Christian wanted to explore Hood.ie life on stage but couldn't get the necessary code in time. So instead Kore spoke a bit about the core idea behind CouchApps. I was also able to add some tidbits about what Hood.ie adds as I recently got a personal demo by Gregor, one of the core developers. We then went through some of the code examples on the site. In general I believe that the coolest thing in CouchDB is the replication protocol, which is why I never understood why Couchbase decided to focus on elastic scaling instead. Thanks to async multi master replication with local storage one can build a whole new class of mobile applications. There is a very high chance that Hood.ie will become a big part of the Liip technology stack.
Symfony2 Rest APII held this session by basically doing a walk through of a client application we are currently building as well as the Liip Symfony2 techtalk edition. Overall I was showing off how to integrate various Bundles: FOSRestBundle, "/>
Truncated by Planet PHP, read more at the original (another 1567 bytes)
#phpuceu
It’s warm, the sun is shining, the sky is blue, you have php nerds ranting around - sounds familiar? Yes, it’s PHP Unconference time! It was the 2nd PHP Unconference in Europe, which took place last weekend at the Free University of Berlin. Thus, we Jimdos didn’t want to miss the opportunity to attend this awesome (un)conference.
There were loads of cool sessions, plus we gave a few ourselves, so here’s our review:

Unfortunately, there was no official pre-party, but our friend Till ‘@klimpong’ Klampaeckel organized a little get-together in the tiniest bar with the slowest barkeeper in Berlin. First day starts - as every unconference - with the voting session. In case you don’t know what’s an unconference: Every attendee can propose a talk or interest and introduce it with a few sentences. After that everybody can vote for the talk he/she would like to hear. This resulted in the following schedule:

We started with Lukas’ talk about PHPCR, which is a set of php interfaces following the JSR-283 standard. There’s also a reference implementation called jackalope. Lukas gave us some very nice insights to the project itself and the goals they want to achieve.
Next up: I talked about “What’s wrong with php?”. Sönke co-moderated a session about puppet, where they talked a lot about anti-patterns in puppet, and why you should also use your existing knowledge about software design when writing puppet. Sidenote: checkout out our puppet-skeleton for testing purposes.
Kore and Gaylord gave a session about behavior driven development and testing in general, to ensure you as developers and the marketing/product people are talking about the same thing when planning a product. We talked a lot about Behat, Mink and the do’s and don’ts in BDD - also this session was kind of a great foundation for Sönkes talk the next day.
Next day started the same as the first, with a voting session. Many of the attendees came up with new ideas for a talk, thus we had some great topics for the 2nd day.

Truncated by Planet PHP, read more at the original (another 7332 bytes)
Our Failure As An Industry
Read more »
