Payflow link invalid input errors

If you are a playflow link merchant and you recently started having “Invalid input” errors from your payment pages, check the source of your form that submits the data to payflow link. Sometime recently (May or June 2013) Payflow began rejecting HTML tags in form field input. I had a “<BR>” tag in my description field, and users were being kicked out with an “invalid input” error.

Android poor battery life, Exchange Services

I had a new Android phone the other day, fresh out of the box; set up my hosted exchange account, and my battery life was measured in hours (as in 5-6 hours). The phone was getting hot all the time and the data arrows were constantly lit. Stopping “sync” or preventing email from syncing in settings solved it, but obviously that’s not a tenable solution for a mobile device.

I logged into OWA and sent logs to myself between the phone and the Exchange server. Over the course of a few minutes there were more than 60 of these:


SyncCommand_OnExecute_Exception :
Microsoft.Exchange.AirSync.AirSyncPermanentException
at Microsoft.Exchange.AirSync.RecipientInfoCacheSyncCollection.OpenSyncState(Boolean autoLoadFilterAndSyncKey, SyncStateStorage syncStateStorage)
at Microsoft.Exchange.AirSync.SyncCommand.SyncTheCollection(SyncCollection collection, Boolean createSubscription, Boolean tryNullSync)
at Microsoft.Exchange.AirSync.SyncCommand.OnExecute()

LogicalRequest :
<?xml version="1.0" encoding="utf-8" ?>
<Sync xmlns="AirSync:">
<Collections>
<Collection>
<SyncKey>613780178</SyncKey>
<CollectionId>RI</CollectionId>
<DeletesAsMoves/>
<GetChanges/>
<WindowSize>10</WindowSize>
<Options/>
</Collection>
</Collections>
</Sync>

Command_WorkerThread_Exception :
--- Exception start ---
Exception type: Microsoft.Exchange.AirSync.AirSyncPermanentException
Exception message:
Exception level: 0
HttpStatusCode: OK
AirSyncStatusCode: Search_TooComplex
XmlResponse:
<?xml version="1.0" encoding="utf-8" ?>
<Sync xmlns="AirSync:">
<Status>8</Status>
</Sync>
Exception stack trace:    at Microsoft.Exchange.AirSync.SyncCommand.OnExecute()
at Microsoft.Exchange.AirSync.SyncCommand.ExecuteCommand()
at Microsoft.Exchange.AirSync.Command.WorkerThread()
Inner exception follows...
Exception type: Microsoft.Exchange.AirSync.AirSyncPermanentException
Exception message:
Exception level: 1
HttpStatusCode: InternalServerError
AirSyncStatusCode: ServerError
XmlResponse:
[No XmlResponse]
Exception stack trace:    at Microsoft.Exchange.AirSync.RecipientInfoCacheSyncCollection.OpenSyncState(Boolean autoLoadFilterAndSyncKey, SyncStateStorage syncStateStorage)
at Microsoft.Exchange.AirSync.SyncCommand.SyncTheCollection(SyncCollection collection, Boolean createSubscription, Boolean tryNullSync)
at Microsoft.Exchange.AirSync.SyncCommand.OnExecute()
--- Exception end ---

AccessState :
Allowed

AccessStateReason :
Global

ResponseHeader :
HTTP/1.1 200 OK

ResponseBody :
<?xml version="1.0" encoding="utf-8" ?>
<Sync xmlns="AirSync:">
<Status>8</Status>
</Sync>

These errors were coming in at the rate of 1 per second, which explains why my phone was hot to the touch.

The solution (for me) was to delete the Exchange account from the phone, delete the phone from OWA, and redo the partnership. If it makes any difference, this time I selected “All” for the length of time to download emails, and I chose manual setup instead of Autodiscover setup. Everything is working normally now.

Jawbone / myTalk Password Breach

Today I received an email from Jawbone explaining that their myTalk customer password list had been stolen by hackers. They say passwords were encrypted, but they do not say how they were encrypted. If they used pig latin to encrypt, then I should immediately go about ensuring that the password I used on the myTalk web site is not used anywhere else. But if they used bcrypt or scrypt with a high work factor and a random salt, I would be somewhat less concerned about this. Not that I wouldn’t check it, but I might do so after lunch, as opposed to, say, right now.

Of course they don’t say what encryption method was used for the passwords, which leads me to believe it leans toward the pig latin side of things, as they don’t feel confident in reassuring their customers that hackers will have to work very hard to obtain the plaintext passwords.

I was going to write to Jawbone and ask them what encryption method they used, but I bet they would first think I’m the guy that stole their data (most of their employees probably don’t even know that most modern encryption methods contain a header of sorts that says what algorithm was used, so the hackers already have this information and us customers do not!), and second they wouldn’t respond anyway, or there would be some line about proprietary information, security concerns, blah blah blah. Jawbone, you should know that once the data leaves your control – encrypted or not – it is no longer proprietary. Your customers have a right to know their relative risk and you should have given us that knowledge right away.

We are a China based imaging professionals / Rick photo editing spam!

After years of putting up with Rick and his digital photo editing services crap I think I have finally figured out a way to block these messages.

Here’s the deal. You run a mail server, and you get spam every single day from Rick offering digital photo editing services. You put “We are a china based imaging professionals” into your spam blocker/body_checks and THE MESSAGES KEEP COMING. You can’t figure out why they are still coming. Your rules are working. You forward yourself one of Rick’s messages and it is blocked. Yet they keep coming from China.

The answer is in the mail log:

Jan 19 19:23:37 dallas postfix/cleanup[21010]: 774CC582A5: info: header Subject: =?GB2312?B?RGlnaXRhbCBQaG90byBFZGl0aW5nIFNlcg==?=??=?GB2312?B?dmljZXM
gLSBQaG90byBDdXRvdXQgLSBQaA==?=??=?GB2312?B?b3RvIFJldG91Y2hpbmc=?= from h184-60-84-203.nwblwi.dedicated.static.tds.net[184.60.84.203]; from=XXXX to=XXXX

I have Postfix report message subject lines in the log because it makes troubleshooting mail problems easier. Look at the Subject: header of this message. It begins with “=?GB2313?B?…” if you take that whole string and put it into Google you get no hits. But if you truncate to just the part I quoted above, you get some very interesting results. GB2312 is the character encoding for Chinese simplified. So Rick figured out that if he encodes his emails using that character encoding but such that the decoded email appears in US English ASCII characters (A-Z|a-z|0-9) then we’ll all get to read about his photo editing services in a language we understand.

Solutions vary – blocking this encoding string in the headers is what I’ll be doing (along with some other encoding strings). There are other articles on this topic once you know what to search for, with various strategies depending on your needs and your email system.

In my case I’m just adding this line to my header_checks:

/^Subject: .*?GB2312?B?.*/ REJECT # No chinese encoding please.

Happy spam-blocking!

Edit: One thing to note. I had a rule set up in Postfix that printed the email subject line in the logs (it’s my mail server, I can do what I want):

/^subject:/ INFO

This line appeared first in my header_checks file, but since only one rule can fire per email header line, after I added this to the top of my header_checks, Rick’s email started getting through again. Sure enough, testing one of his headers with:

postmap -q 'Subject: =?GB2312?B?' regexp:/etc/postfix/header_checks

resulted in:

INFO

instead of:

REJECT

because only the rule that writes the subject to the log was being run. So I moved that rule to the bottom of header_checks. Now the messages with Chinese encoding are blocked (REJECT) and if any mail does get through, the subject line is written to the logs.

1099-K and transaction date or settlement date

I believe I’ve uncovered an accounting nightmare for merchants who receive 1099-Ks. I noticed that some of my monthly totals for 1099-K were off by small amounts vs. my (very) detailed monthly accounting. Turns out, the 1099-K’s appear to have been generated based on transaction date, while my reporting is traditionally based on settlement date (i.e., the date the merchant receives the money).

I would consider the settlement date to be the constructive date of receipt, because that is the date the money is transferred to my account. However, the credit card processors are handling this based on transactional data, which causes irregularities because all my bank statements show the income in the subsequent month.

Since it is not possible for a merchant to make such a logical point to the IRS and be understood, the only solution appears to be to edit the dates of the offending transactions to reflect the month (and year) reported by the 1099-K.

Way to go, people.

SPF records and Payflow Link

EDIT – Update January 3, 2013: There was a bug in my previous SPF record below, that included “ip4:96.0.0.0/1”. That should have read “ip4:96.6.0.0/11”. Unfortunately the initial incorrect record allowed half the Internet to pass SPF tests, so it would be better to use the corrected version or edit this list to suit your tastes. The SPF record below has been corrected to reflect this change.

_________________________________________

If you use Payflow Link and you’ve recently implemented an SPF record with a hard-fail (“-all”), you and your customers may periodically stop receiving transaction confirmation messages from Paypal’s servers.

Your mail logs will hopefully show an error like:

2011-10-22 17:09:12 H=mx0.phx.paypal.com [66.211.168.230] F=<REDACTED EMAIL> rejected RCPT <REDACTED EMAIL>: SPF: 66.211.168.230 is not allowed to send mail from REDACTED DOMAIN

So the question is, what IP address or block should you add to your SPF record to allow PayPal’s servers to send mail on your behalf? PayPal has a range of IP addresses of course; their transaction servers are different from the email servers (apparently).

The correct answer is here, in PayPal’s list of “IP Addresses for PayPal servers”:

https://ppmts.custhelp.com/app/answers/detail/a_id/92

Since there are so many IP ranges here, my SPF record was getting too long. So I used the include: directive and created an SPF record for a subdomain (spf-paypal.MYDOMAIN.com):

“v=spf1 ip4:173.0.80.0/20 ip4:64.4.240.0/20 ip4:66.211.160.0/19 ip4:216.113.176.0/20 ip4:118.214.0.0/15 ip4:92.123.0.0/24 ip4:96.6.0.0/11 -all”

And then in my main domain SPF record, I added “include: spf-paypal.MYDOMAIN.com”. This allows me to easily make edits to the PayPal portion of the SPF record and keeps it separate from my main SPF record. PayPal makes IP address changes on average perhaps twice a year, so if you use PayFlow to send transaction receipts to your customers on your behalf and you maintain SPF records for your domain, this is something you will encounter.

At the bottom of PayPal’s post, there is an option to subscribe to the answer so you can be alerted when it changes. Unfortunately they do not show a changelog; they just update the answer, so every time they change it you have to go through and compare every IP range with your SPF record, but at least you can be alerted when there is a change.

Previously, I had looked back through some transactions and (since a recent IP address update) all of them seem to be from 66.211.x.x, so my ham-fisted approach was to add:

ip4:66.211.0.0/16

to my SPF record, to allow 66.211.0.0 – 66.211.255.255.