All talk but no code...

makes lulalala a bluff boy

My Presenter Library for Rails

| Comments

Naming view helper methods in Rails can be difficult. What do you think about the following helpers?

auction_title(@auction)
board_title(@board)
category_title(@category)

Since all helpers live under the same scope, they all need to be named differently to avoid name collision. This can become tedious as the number of models grow.

For me personally, these two issues always bugged me when creating a helper method:

  1. I don't know where to put the helper method.
  2. I worry about name collision.

Reinventing the wheel

People thought of ways to organize these helper methods, which are now commonly called presenters, decorators or exhibiters.

One school of thought uses decorators to append functionalities on top the ActiveModel, draper, display_case and active_decorator being the three most well-known solutions.

Built-in helper methods all reside within view-context objects. So for presenter to work it needs access to the view-context object. However passing it for every helper call can be tedius. I liked how draper and active_decorator grabs that view_context for you, so you won't have to keep passing it to presenter when calling it.

However I see that its effort in decorating the model object can easily leak. For example would you remember to also decorate the association objects?

I wanted something else instead, a presenter object which contains only the helper methods. I add a presenter() method in the model object to access the presenter object. No decoration and less issues. The presenter() object also takes care of caching the presenter object, so you only instantiate a presenter per model once.

So I made my changes on top the active_decorator, and called it lulalala_presenter (all the good names are taken already).

Usage

First install by putting this in the Gemfile:

gem 'lulalala_presenter'

Say you want to add a title() helper method for auction, which takes care of truncation and html cleansing. I simply create a AuctionPresenter class. Remember to subclas it with LulalalaPresenter::Base:

# app/presenters/auction_presenter.rb
module AuctionPresenter < LulalalaPresenter::Base
  def title
    h.truncate(model.body, length: 40)
  end
end

And then in the view I call it like this

<%= @auction.presenter.title %>

Less magic

I think it is good to start with less magic, but allow user to add magic if they want more convenience.

By default you use h to access all the build-in helper methods, and model to access the model object. If you are lazy you can choose to add stuff in your presenter so you can type less. For example you can delegate a few model attributes:

delegate :body, to: model

You can probably also automatically delegate everything to models using other delegation libraries.

Conclusion

At the end I quite like my results. For the top example, now I have

@auction.presenter.title
@board.presenter.title
@category.presenter.title

Having to type .presenter when calling helpers isn't so bad as I first thought. This makes it very obvious that the method belongs to presenter, so you won't ever mix up a helper methods with model methods.

My two goals are also fulfilled. If I need an og_image_tag helper for my auction, I immediately know I should put it in the AuctionPresenter, and I can name it without worrying about prefixing model names.

Apply SEO-friendly url only to show path

| Comments

Rails developers often use to_param to add more information to the url. Taking this very page as an example:

http://logdown.com/account/posts/290439-seo-and-to-param

We can clear get an idea of what this page is about. Therefore search engines favours this kind of urls.

However, if we just override to_param, we would also see other urls getting changed. One most notable example would be edit path:

http://logdown.com/account/posts/290439-seo-and-to-param/edit

I think this can cause issues as the seo-friend param can potentially get very long. Then it would be difficult to notice the edit in the url. Also it may not be that useful to do SEO on action urls (rather than the content urls). We probably will never want edit page to appear on search engines.

I think instead, we should conditionally do SEO on urls that needs it. One way to do this is to have our own version of the url helpers. For example, we can define post_url and post_path like this:

# Override default generated Rails route helpers.
module CustomUrlHelper
  def post_path(target, options = nil)
    target = "#{post.id}-#{post.title[0..60]}" if target.respond_to?(:title)
    super(target, options)
  end
end
Rails.application.routes.url_helpers.send(:include, CustomUrlHelper)

This means only show/delete/update paths will be SEO friendly. Other paths (especially custom actions) will be unaffected.

Rails 4.1 upgrade and FactoryGirl.create with association

| Comments

I was trying to upgrade from Rails 4.0 to Rails 4.1. Some specs broke, and I found a small behaviour change.

So my user has_one profile.

user.association(:profile).loaded? # false
FactoryGirl.create(:profile, user: user)
user.association(:profile).loaded? # false

Prior to my upgrade, the profile association will not be loaded before/after FactoryGirl.create call.

However after Rails 4.1

user.association(:profile).loaded? # false
FactoryGirl.create(:profile, user: user)
user.association(:profile).loaded? # true

This means I have to be careful to not have stale profile in tests, calling reload more often.

P.S. my factory_girl gem has remain unchanged in version 4.4.0 and factory_girl_rails gem in 4.4.1. So this is solely due to Rails behavior change.

從 sqlite 轉換到 postgresql

| Comments

這個週末嘗試把 sqlite 的資料轉移到 postgresql 上,結果遇到許多坑。

sqlite 的 data dump 其實需要更改不少地方,才能用在其他資料庫。這部份很容易犯錯,所以我想找其他方法。

嘗試過的方法

我先是在 RailsCast 上看到 taps 這個 gem ,能夠透過另開 server 作到讀取資料再匯入的功能。可惜這個 gem 已經沒有維護了,自己嘗試修問題都是以失敗告終。

接下來我使用 pgloader ,這似乎能直接匯入資料進入 pg ,可是最近的版本似乎有 UTF8 的問題,我就算直接把 master 抓下來手動編譯,還是無法成功解決這問題,故放棄。

心灰意冷時,我突然想到,以 SQL 不行,但是既然 ActiveRecord 能夠支援各種資料庫,那麼只要把舊的資料以 yaml 格式備份出來,然後在新的資料庫給塞回去,就可以了。找了一下發現 active-dump 這款 gem ,最後作了一些調整,終於成功的轉移資料庫。

注意的地方

  • 有使用 foreign key 的話,記得依照 key 的 dependency 把匯入的順序調整,被 Key 指向的 table 移到前面。
  • 匯出前也要先把 orphan record 先刪除(就是有 foreign key 指向已經不存在的資料時)
  • 匯入前,先轉換資料庫然後跑 migration
  • 匯入後,每個 table 的 id 並不會自動設成從目前最大的 id 開始繼續遞增。必須跑類似 ALTER SEQUENCE product_id_seq RESTART WITH 1453; 來對每個 table 一一作設定,不然的話可能新增的 record 會從前面沒有的 id 開始建立。
  • 以前在 MySQL 或是 Sqlite 時似乎沒有指定 order 就會以 primary key 自動作排序。但是在 Postgresql 我發現丟回來的東西順序不是這樣(我看到的是相反的 id desc)。這代表 query 都得手動加入 order(:id) 才能確保跟以前一樣以 id desc 排序。

後記

資料轉移完以後,我才偶然發現 sequel gem 似乎也有提供類似的轉移功能,因為這款 gem 很有名,所以很有可能比以上我使用的方法更可靠好用。這就等其他人來試試看報告一下了。

Middleware模式的隨想

| Comments

最近想寫一個 email domain 的檢查器,想要提供 wrapper ,讓使用者能包一層 cache 或是給 testing 用的 dummy 。這個時候我就想到了 Rack middleware 的那種模式。

只不過我想說這麼簡單的程式庫,做出 middleware DSL 來是否太過浪費,所以就想說不要做成 middleware 模式那樣剝洋蔥的感覺。所以就打算平面化,用 array 把 wrapper 存起來,每個 wrapper 都有一個 valid? ,依序呼叫後有 true 就提前 return ,false就呼叫下一個。

實做後,發現我忘了應該要 cache miss 跟真正的檢查後,再把東西存起來。要這麼作就只能真正執行檢查後,再迴圈跑每個 wrapper 的另一個方法去存起來。這個時候才發現 middleware 的精妙之處。首先同一個方法就會同時負責 input 跟 output 。然後測試也比較簡單,各個 wrapper 可以獨立寫測試。用我的土炮方法,就得把 wrapper 套在 core class 上進行 integration test,並且確認裡層的東西在 cache hit 時就不會呼叫。

感覺自己因為想減少 method 呼叫層次,似乎只是一種 premature optimization 囧。

Payment Notification Receiver

| Comments

When I am integrating offsite payments in ActiveMerchant, I always need to provide an endpoint to receive payment notificaiton from gateway providers. After a couple of projects, I have come up with a best-practice to do it, and I thought this might be useful for others.

I have a PaymentNotification class solely to record notifications. This is because I want to decouple the persistence of notification from the order status change. If something goes wrong when I change Order, the notification will still be safely persisted in the database for later analysis.

As you can see, the controller action is pretty light. It only saves the notification. The notification request is setup so it contains order_id and payment method.

The interesting part is the PaymentNotification model. It has a few columns for persisting different information (mostly serialized). It is also linked to Order, for easier analysis.

The process method is to check the notification's validity and trigger order status update. It is to be called separately (maybe via a scheduled job). A few kinds of custom exceptions may be raised during process, because they are handled differently. Either way, errors will be recorded in the errors column.

Any improvement is welcomed :D

歐付寶有 < 跟 & 就會認證失敗

| Comments

歐付寶有個機制檢查 POST 內容是否可信,先是把表單所有的資料串起來,然後以 url encode 以後,再用 MD5 跟一組共同秘密加密起來。要是你產生的值跟他伺服器產生的一樣,就算認證通過。

最近發現如果表單有值是 & 或是 < 的話,認證就會失敗。

因為之前也有發生括號符號我們有轉成 %28 但是他們那端沒有轉,導致認證失敗的錯誤。所以我自己也測試了兩種情況:把 & 轉成 %26 或是不轉。兩種情況都認證失敗。

所以就寫信問客服並附了以下的 form 當參考:

<input id="TradeDesc" name="TradeDesc" type="hidden" value="&lt;&amp;" />
<input id="CheckMacValue" name="CheckMacValue" type="hidden" value="4A573F3F559226072079E3758B3D8E1A" />

結果來回了幾封信確認後,客服給了我以下答覆:

<input id="TradeDesc" name="TradeDesc" type="hidden" value="&lt;&amp;" />
一般來說POST資料時不會post到html編碼
請勿使用html encode,
另外,TradeDesc 及 ItemName 請避免使用特殊符號等字元以及html tag 等符號
請您再試試
THX.

當下就有點怒的回信,因為 html 裡面把 & 都轉成 &amp; 是基本知識了,瀏覽器 POST 資料出去時應該會自動轉回 & ,金流端不會接收到 &amp; 這種 escape 字串。而沒有對 < 作 escape 的話,瀏覽器能不能運作都是問題呢。

API 文件並沒有標明不能特別限制特殊符號,經過測試其他特殊符號都可以使用。但是就只有 <& 不行。是說這兩個符號也沒有特殊到哪裡去,要是英文商店用個 & 字元也是很正常的吧。

所以就回了信,說明 &amp; 這 html encode 不會對你們公司 server 收到的資料有所影響,並附上我產生認證碼的每個步驟,希望對方工程師能發現是哪個步驟開始相異。

結果對方客服就給我一個跳針回應:

TradeDesc 及 ItemName 請避免使用特殊符號等字元
THX.

明明是可以解決的問題(只修改我方程式),但是卻直接用這種方式迴避。到底客服有沒有把我的提報轉給工程師呢,或者工程師因為什麼原因不想研究這個問題呢?我原本只是想改善開源的 active_merchant_allpay 串接介面,但是這樣的回應~還是不要熱臉貼冷屁股好了。

ActiveMerchant / offsite_payments gem needs to improve

| Comments

ActiveMerchant is a very useful library for connecting to payment gateways. However it is also quite dated, with many flaws, and lack of proper documentation in one part (offsite-payment). In my experience, I feel it can be improved in the following ways:

The name "integration" in confusing, and makes it diffcult to search online. The gem should be split into two, one for on-site payment (gateway/billing module), and one for off-site payment (integration/Billing::Integration module). This is because the two are very different ways of paying. It can be confusing when you search for solutions but it turns out to be not what you want. Seems Shopify is alreadying doing that, splitting the offsite-payment module into its own gem. Good move!

The semantic of common fields in form helper need to be defined. Otherwise implementors will choose different names for the same concept, greatly reducing the interchangeability between gateways. Some consistency issues should also be fixed. For example in Paypal we pass our order id as the "order" field, and Paypal notification will return that as item_id attribute. This mismatch is confusing.

The gross() currently does not specify a return type. Paypal returns a string. Base class says it should "the money amount we received in X.2 decimal". Though there is a amount() method for returning a money object, the gross() method should still specify the return type to avoid inconsistencies when switching gateways.

The base framework code should be separated from gateway implementations. Users usually just want two or three gateways out of the 50 implementations. Having a separate repository per gateway also makes it easier to document gateway specific changes and settings in separate readme files.

There should be a check() method in the notification. It acts as the central method to check if the notification is valid. Currently there are many which implemented the acknowledge() function. However not all providers provide an API to verify the notification. With the generalized check() method it can call acknowledge() if it is implemented.

Adding hooks for form helper would be helpful. Many gateways require post-processing such as adding a checksum field. A hook would allow the interface to automatically call these post-processing instead of the developer having to remember calling them in view.

Markdown 或是 LATEX

| Comments

看到一篇文章(http://yihui.name/en/2013/10/markdown-or-latex/),分析寫作時該選擇 Markdown 或是 LATEX 呢。

Markdown 跟 LATEX 其實要解決的是不同的問題,也就是你要產生需要排版有頁數的實體文件,還是就只是一長串的文件?

LATEX 要解決的一個核心問題就是必須解決分頁時放置圖片或是表格的問題。如果沒有分頁,那麼這些東西自然依照文字排放往下排即可,可是因為這些東西要是正好排在分頁邊緣時,不能腰斬只顯示一半。此外,因為分頁所以排版上也冒出許多限制,這些都是 LATEX 的強項。

所以分辨的原則就是,如果你需要產生有分頁的文件,而且有許多漂移元素時,就選 LATEX 吧。不然就選你感覺比較舒服的即可,因為簡單的文件是可以簡單轉換格式的。

TrueCrypt 官網突然說自己不安全

| Comments

TrueCrypt 是一個開源免費的加密軟體,能製作加密過的虛擬硬碟,供使用者存放資料。

就在昨天五月二十四號,它的官網突然開始轉址回 sourceforge ,並宣布自己的加密有被破解的風險,而 TrueCrypt 本身的開發也終止,主要原因是因為微軟已經停止 XP 的支援,而其之後的各個作業系統都有 TrueCrypt 的替代品。比如說 Windows 使用者應該轉向使用微軟的 BitLocker。而最新的版本7.2也公開,這是一個只能解密不能加密的版本,意在讓使用者解密後轉移資料至其他軟體。

官網紅色大字「WARNING: Using TrueCrypt is not secure」十分格格不入。

先前四月的時候 TrueCrypt 的程式碼被第三方做了基礎檢查檢查,結果並無發現有後門(網站)。所以在現在突然宣布自己不安全還蠻匪以所思的。

TrueCrypt 本身雖然是開源軟體,不過似乎沒人知道後面的作者到底是誰。所以也引來許多陰謀論,比如說 TrueCrypt 本身是就是美國政府旗下製作的陷阱。或者這是作者跟NSA妥協下的結果。到目前為止似乎沒人能證明這不是網站被駭客修改。不過由於這次的改變十分細緻,比如說程式碼換了授權,而且還做了修正變成只能解碼。所以感覺不像是外來駭客會作的事情。