Contents

Data privacy

The app is different from a conventional type of spam filtering software where user specified spam data are uploaded to its own App server, analyzed. the machine learning is performed to generate a spam filter. The spam filter resides on the App server. And the App server does the spam filtering. This is a big problem in terms of data privacy.

But this app is completely a client side app.
The spam data are inside iOS devices. Those data never go outside of devices except iCloud option is enabled(this option is not currently recommended).
Machine learning algorithm is embedded in the app, so the machine learning is done in a device and a generated spam filter is also in the device. The spam filtering is done in the device.
Starting from specifying learning data to generatring a spam filter is done in the device, and it's done without using any 3rd party software parts. In terms of data privacy, it completely keeps privacy.

In addition to spam filtering, the app supports rule based mail filtering. In this case, also user specified rules are stored in the device.
The data never go outside except iCloud option is enabled(this option is not currently recommended).

Conventional spam filtering

The App's spam filtering


Notice:Since iOS doesn't allow a daemon process, the spam filtering implementation should be either of above two.

Concerning other data privacy, please read How a spam filter is generated? and privacy policy.

Introduction

How a spam filter is generated?

To make you sure how data privacy are kept, we'll explain how email specified as learning data are anaylzed and transformed to a spam filter through machine learning.

At first, Ngrams(specifically unigram, bigram, trigram) are generated from email specified as learning data. Ngrams are character based.
Secondly, features which are effective in spam judgement are selected among generated Ngrams.
Thirdly, each feature weight is calculated based on how effectively it judges spam through machine learning.
Finally, pairs of {selected feature, weight} are generated to become a spam filter.

During this process, the app doesn't access any external software modules, and doesn't use any 3rd party software modules.
Ngrams are generated in memory, which will be deleted after feature selection.
A generated spam filter are stored in the device's storage.

Thus the user data thoroughly stay private.

Spam Filtering

There are server side spam filtering and local spam filtering.
Server side spam filter uses blacklist/whitelist,rule based algorithm, or some kind of machine learning algorithm.
But it cannot detect all spams because spam or ham judgement depends on each one's tastes.
Local spam filtering is essential if you want to filter out annoying messages that are particular to you.
The app can do a local spam filtering.
The app has a built-in machine learning categorization engine and generate an effective spam filter.

Design Your Own Spam Filter

The app generates a spam filter by learning from spam and ham messages in INBOX specified by you.
After learning a certain amount of data, set the spam filter active at "account setting", incoming messages will be categorized into spam and legitimate ones.
The spam filter represents your tastes since you specified which messages are spam or legitimate.
The spam filtering gets the most powerful when the local spam filter gets along with the server side filter.

Server side Spam filtering's strength

The setting of the server side spam filtering's strength should be low or middle.
If you set it high, some legitimate messages might be treated like spams.
General spams should be filtered out at the server side filtering. personal spams should be filtered out at the client side.
But a mail service provider sometimes does not offer the spam filtering's strength adjustment option.

Connect to a Mail Server

"New Account" screen appears after SpamBlocker's first start up.
Enter your mail address, password and tap "connect" button,
the activity indicator will appear on the navigation bar.
After a while, the monitor screen will show up.
If the login name is not your mail address, you have to enter a login name in "account setting" screen.
To go to the account setting, tap manual button which appears on the navigation bar.

Account Addition
Account Setting
⇒

If you don't know the login name/password, contact an organization or personnel in charge of the mail server to get a hostname, login name(user id), password.


You don't have to specify port no.
Port no is to be alerted according to SSL setting.

Account information

The account information set to the app is exactly the same as one in other mailers. If you want to use the same IMAP mail server, set its account information to the app. The connection will never fail.


How to cope with the connection error

How to cope with 'Network Error'

After once the connection is successfully established, 'Network error' might occur due to an 'IMAP server' network condition.

Network Error

If you choose 'Offline', long-press on the table cell of 'Account' list to go back to online. If only one account is registered, stop running MailCat and re-invoke it.

Back to Online

Spamfilter

To generate your spam filter, take the following steps.



Phase one:   Generate a Spam Filter

  1. Tap "Edit" button.
  2. Pick up several mail messages by tapping table cells.
  3. Tap "SPAM" button on the command bar.
  4. Likewise pick up mails, and "HAM" button on the command bar.
  5. ""Pull to Refresh" or "Long Press on Navigation Bar" triggers Machine learning, then a spam filter will be generated.
  6. Checks the learning results. Correct learning errors and "Pull to Refresh" or "Long Press on Navigation Bar" again.

Repeat the steps until the number of spams and hams reaches 16.

Select learning data
Selected learning data
Select mails, tap "SPAM" button on the command bar. Red circle appears at the left of each selected mails. Green one appears in case of ham.
Learning
Do "Pull to Refresh" or "Long Press on Navigation Bar". Learning starts.Red and green circles turned gray ones. "HAM" or "SPAM" icon is added at the left of a sender name.

Phase two:   Train Your Spam Filter

Learning error correction can be done by tapping "Edit" button, repeating the same process. The spam filter is set off by the default setting. After learning more than 16 spams and 16 hams, the app is able to judge which mail is spam or not. To start spamfiltering, the spam filter switch should be set on at account setting.

  1. Check the spam filtering results on Monitor.
  2. Correct errors and do "Pull to Refresh" or "Long Press on Navigation Bar", re-learning starts and your spam filter gets smarter.

Repeat the steps until no errors are found.

Check results
Correct results
Check spam filtering results. If filtering results are against your taste, tap "Edit" button, correct them.

Re-Learning
Check again
Do "Pull to Refresh" or "Long Press on Navigation Bar" after the correction. Check to see if the results are satisfying, again.
Correction of Learning Data/Spam filtering results

A Green mark or red mark turn a gray one to indicate learning is done on each message.
If the learning is done, the icon such as a happy icon or a spam icon appears, otherwise either of the following icon appears.



The first example shows that a legitimate mail is mistakenly categorized into a spam mail.
The second one shows the app cannot decide whether a mail is legitimate or not.
The second case is commonly observed during the initial learning phase, this phenomenon will gradually vanish as the learning process continues.
The first example is frequently observed if one of the similar mail messages is specified as a SPAM and another one is specified as a HAM. the app dislikes the inconsistency.
The app supports incremental learning, so learning mail messages can be added or removed anytime as long as the messages can be seen on Monitor screen.
Even if a mail message is specified as a HAM, it can be re-specified as a SPAM.
Thus, if you met the above two example, you could re-specify the mail message then can do re-learning.

For instance, each time you tap the selection mark, it turns the status of the mark into HAM, selection, no selection ,... in "Editing" mode.
You can choose any status. Thus you can modify "Learning Data" status.

Learning data should be balanced

Numbers of HAM and SPAM messages should be balanced.
It's OK if it's not extremely unbalanced.
If it's balanced, you can get the better results.
If you learn more than 200 learning messages which are spam&ham combined figure, a generated spam filter can work practically.
A spam filter generated from 10 learning messages may work, but its predictability is not trustworthy.


Apply Spamfilter

The Spamfilter stays inactive unless 'Spamfilter' switch is on.
Each account has 'Spamfilter' switch.
Visit "Account Setting" at "Setting".

Mail Rules

There are 4 conditional fields for mail rules --- sender, subject, message, compound.
Mail rules are only applied for 'INBOX'.

To edit or create a mail rule, tap 'Edit' button on a Mailbox list. The selection menu appears, choose 'Rule'. Mail rules for ‘Sender’ will appear.
To switch a view, tap a Tab bar.

To add a rule for 'Sender', 'Subject', ‘Message’, pull down the table view, then an empty cell will be added with saying ‘Tap to Edit’.

To delete a rule, swipe the cell to the left. Tap the delete button to be appeared.

To create a spam rule, go to Setting and tap "SPAM Rules", the rest is the same.

Compound Rule

Rules moved from 'From', 'Subject', ‘Message’ are combined by 'AND', ‘OR’, 'NOT' operator into a compound rule.
To move a rule, swipe right a table cell and tap "Comp." button. The moved rules are invalidated until they are combined by 'AND', ‘OR’, 'NOT' operator.
To make a compound rule, tap the edit button on the Navigation bar, 'AND', ‘OR’, 'NOT' on the command bar will appear.

The following symbols are used for Compound rules.

Symbols used in Compound rules

Logical operator

∧
AND
∨
OR
¬
NOT

Field mark

F.
Sender address
S.
Subject
C.
Content of Message
Compound Rule Examples

Compound Rules can be nested infinitely. But the acceptable level is 3 or 4 from a standpoint of human's cognitive capability.
Here are two practical examples of compound rules.

First example is 3 level nested compound rule, but it is relatively simple.
Second one is to absorb ambiguity. It searches mails with the subject like "ten patients" or "flu patient"

When you cannot express a search condition by a simple rule , use a compound rule.


  • (C.finance ∨ C.money ∨ C.stock )∧ ¬(@.daiwa ∧ @.nomura)

  • (S.10 ∨ S.ten ∨ S.Ten) ∧ (S.Flu ∨ S.flu ) ∧ S.patient

    • Apply Mail Rules

    The Mail rules filter the incoming messages, then the spamfilter will filter the mail rule filtered messages.
    Thus mail rule filtering comes first before the spamfiltering.
    The mail rules don’t work unless 'Mail Filter' switch is on.
    Each account has 'Mail Filter’ switch.
    Visit "Account Setting" at "Setting".
    There are two kinds of the mail rules.
    One is globally effective which means effective in all accounts, the other is locally effective which means effective in a particular account.
    The globally effective mail rules can be specified in system mailboxes at the unified mailbox list.

    Mailbox Attribute Editing

    In Mailbox editing mode, the following operations are allowed.
    The facitility is only provided for MailCat, MailKit.

    As for "Attribute setting", a mailbox is changed into a system mailbox by setting it an system mailbox attribute.
    A system mailbox is a candidate to the "Quick View".
    "Quick View" is one of views which can be seen through the rotation by tapping the navigtion bar on Inbox/Monitor.
    ""Attribute setting" is to be used for "Mailbox mapping".

    Attribute setting

    iCloud

    If the iOS device is signed in iCloud, iCloud option can be selectable at Advanced Setting.

    The Machine learning data and rule data are uploaded to iCloud when the App goes under background status by hitting Home button of the iOS device or it starts filtering.
    The iCloud synchronization sometimes delays.

    When the iCloud synchronization doesn't seem to occur, tap Rebuild iCloud Store at Advanced Setting. It may take a few minutes.
    During rebuiding iCloud Store, there is no machine learning data and rule data available in the App.

    If iCloud is selected, the alert message may be displayed, saying,

    Overwrite deletes old iCloud data, copy local data to iCloud.
    Merge copies only different part of data to iCloud.
    Skip is just a mode change.

    If the iOS device is signed out from iCloud, iClould is not available for the App.

    If the iCloud option is selected, the machine learning data and rule data are synchronized over iCloud and shared between MailCat, MailKit, SpamBlocker.

    To exit from iCloud, tap iCloud whose background cell color is red at Advanced Setting. Data backup to local can be done by selecting Overwrite or Merge.

    For some reason, iCloud store is corrupted, and cannot be auto-recovered,
    Take the following steps to restore iCloud store from the local store.

    1. Deletes MailCat iCloud store at iCloud Storage management section of Setting App.
    2. Tap iCloud whose background cell color is red at Advanced Setting
    3. Tap iCloud whose background cell color is blue at Advanced Setting. Then choose Overwrite

    FAQ

    Network Error

    A "Network Error" alert may occur when the app starts.
    In that case, tap the retry button to try to connect to a mail server again, or tap the cancel button to let an account offline.
    The offline account can be changed to the online state if you long-press the account on the account list or setting the account switch off to on at the app's setting.

    Account off⇒on
    long-press to online

    Spam filtering doesn’t work in spite of machine learning efforts

    The reason is one of the followings.

    Spam filtering of the received mail messages(messages in Inbox)

    Confirm the setting or set the option properly.

    A mail message is not categorized into the right one

    The reason is that the number of learning data is too few.
    The problem is solved by adding more learning data.
    SpamBlocker searches mails that match with registered mail addresses in the Address Book among the received mails in Inbox, marks them as HAM(a legitimate mail) if there are no legitimate mails specified.
    But it will ends up specifying similar mail messages as legitimate mails and decrease the machine learning performance.

    Specifying learning data in the monitor mode will improve the situation.

    Miss-specified learning data and the machine learning had done

    The situation is recoverable.
    SpamBlocker allows a user to re-specify the learning data’s category(HAM or SPAM).

    Rule filtering speed is low

    　If message rules are specified, the rule filtering speed goes down.

    iCloud data is not synchronized.

    Look at iCloud.

    iCloud data is corrupted

    Look at iCloud.

    The App is not responsive(runs slow)

    The reason is that memory for the app is running short or the network environment is being poor.
    To solve memory shortage, terminate SpamBlocker and restart it.
    If the situation remains same, terminate other apps which are not in use.
    If it doesn’t improve the memory shortage, power off the iOS device(iPhone/iPad) and power on the device.
    If SpamBlocker still runs slow after these efforts, it is the network issue.
    Use Spamblocker under the Wi-Fi environment.

    Support Information

    If you have any questions or requests or find bugs, please send email to:

    support_seimitsu@jcom.home.ne.jp