Disruptor with multiple consumers so that each event is only consumed once?

There are 2 approaches, the first is to use the WorkerPool. The second it to use the ‘striped’ EventHandler approach described below.

If we have 4 handlers and assign each an ordinal (0 through 3), then the consumer need only do a modulo operation using the sequence number and the number of consumers and compare it to its ordinal value.

public final class MyHandler implements EventHandler<ValueEvent>
{
    private final long ordinal;
    private final long numberOfConsumers;

    public MyHandler(final long ordinal, final long numberOfConsumers)
    {
        this.ordinal = ordinal;
        this.numberOfConsumers = numberOfConsumers;
    }

    public void onEvent(final ValueEvent entry, final long sequence, final boolean onEndOfBatch)
    {
        if ((sequence % numberOfConsumers) == ordinal)
        {
            // Process the event
        }
    }
}

Some would ask if one consumer takes too long on a transaction it will block all queued on that ordinal. Technically this is possible but one must consider that the batching effect then kicks in thus saving cost for those behind. With this approach the concurrency costs are so low you may find even small stalls are less costly than a queue based alternative.

DisruptorFrequently-Asked-Questions
并发框架Disruptor译文

面向对象六大原则

单一职责原则(Single-Resposibility Principle)。

“对一个类而言,应该仅有一个引起它变化的原因。”本原则是我们非常熟悉地”高内聚性原则”的引申,但是通过将”职责”极具创意地定义为”变化的原因”,使得本原则极具操作性,尽显大师风范。同时,本原则还揭示了内聚性和耦合生,基本途径就是提高内聚性;如果一个类承担的职责过多,那么这些职责就会相互依赖,一个职责的变化可能会影响另一个职责的履行。其实OOD的实质,就是合理地进行类的职责分配。

开放封闭原则(Open-Closed principle)。

“软件实体应该是可以扩展的,但是不可修改。”本原则紧紧围绕变化展开,变化来临时,如果不必改动软件实体裁的源代码,就能扩充它的行为,那么这个软件实体设计就是满足开放封闭原则的。如果说我们预测到某种变化,或者某种变化发生了,我们应当创建抽象类来隔离以后发生的同类变化。在Java中,这种抽象是指抽象基类或接口;在C++中,这各抽象是指抽象基类或纯抽象基类。当然,没有对所有情况都贴切的模型,我们必须对软件实体应该面对的变化做出选择。

Liskov替换原则(Liskov-Substituion Principle)。

“子类型必须能够替换掉它们的基类型。”本原则和开放封闭原则关系密切,正是子类型的可替换性,才使得使用基类型模块无需修改就可扩充。Liskov替换原则从基于契约的设计演化而来,契约通过为每个方法声明”先验条件”和”后验条件”;定义子类时,必须遵守这些”先验条件”和”后验条件”。当前基于契的设计发展势头正劲,对实现”软件工厂”的”组装生产”梦想是一个有力的支持。

依赖倒置原则(Dependecy-Inversion Principle)。

“抽象不应依赖于细节,细节应该依赖于抽象。”本原则几乎就是软件设计的正本清源之道。因为人解决问题的思考过程是先抽象后具体,从笼统到细节,所以我们先生产出的势必是抽象程度比较高的实体,而后才是更加细节化的实体。于是,”细节依赖于抽象”就意味着后来的依赖于先前的,这是自然而然的重用之道。而且,抽象的实体代表着笼而统之的认识,人们总是比较容易正确认识它们,而且本身也是不易变的,依赖于它们是安全的。依赖倒置原则适应了人类认识过程的规律,是面向对象设计的标志所在。

接口隔离原则(Interface-Segregation Principle)。

“多个专用接口优于一个单一的通用接口。”本原则是单一职责原则用于接口设计的自然结果。一个接口应该保证,实现该接口的实例对象可以只呈现为单一的角色;这样,当某个客户程序的要求发生变化,而迫使接口发生改变时,影响到其他客户程序的可能生性小。

良性依赖原则。

“不会在实际中造成危害的依赖关系,都是良性依赖。”通过分析不难发现,本原则的核心思想是”务实”,很好地揭示了极限编程(Extreme Programming)中”简单设计”各”重构”的理论基础。本原则可以帮助我们抵御”面向对象设计五大原则”以及设计模式的诱惑,以免陷入过度设计(Over-engineering)的尴尬境地,带来不必要的复杂性。

Java gotchas

https://www.owasp.org/index.php/Java_gotchas

Equality

Object equality is tested using the == operator, while value equality is tested using the .equals(Object) method.

For example:

String one = new String("abc");
String two = new String("abc");
String three = one;
if (one != two) System.out.println("The two objects are not the same.");
if (one.equals(two)) System.out.println("But they do contain the same value");
if (one == three) System.out.println("These two are the same, because they use the same reference.");

The output is:

The two objects are not the same.
But they do contain the same value
These two are the same, because they use the same reference.

Also, be aware that:

String abc = "abc"

and

String abc = new String("abc");

are different. For example, consider the following:

String letters = "abc";
String moreLetters = "abc";
System.out.println(letters==moreLetters);

The output is:

true

This is due to the compiler and runtime efficiency. In the compiled class file only one set of data “abc” is stored, not two. In this situation only one object is created, therefore the equality is true between these object. However, consider this:

String data = new String("123");
String moreData = new String("123");
System.out.println(data==moreData);

The output is:

false

Even though one set of data “123” is stored in the class, this is still treated differently at runtime. An explicit instantiation is used to create the String objects. Therefore, in this case, two objects have been created, so the equality is false. It is important to note that “==” is always used for object equality and does not ever refer to the values in an object. Always use .equals when checking looking for a “meaningful” comparison.

Immutable Objects / Wrapper Class Caching

Since Java 5, wrapper class caching was introduced. The following is an examination of the cache created by an inner class, IntegerCache, located in the Integer cache. For example, the following code will create a cache:

Integer myNumber = 10

or

Integer myNumber = Integer.valueOf(10);

256 Integer objects are created in the range of -128 to 127 which are all stored in an Integer array. This caching functionality can be seen by looking at the inner class, IntegerCache, which is found in Integer:

 private static class IntegerCache 
 {
   private IntegerCache(){}
   
   static final Integer cache[] = new Integer[-(-128) + 127 + 1];
 
   static 
   {
     for(int i = 0; i < cache.length; i++)
     cache[i] = new Integer(i - 128); 
   }
 }
    
 public static Integer valueOf(int i) 
 {
	final int offset = 128;
	if (i >= -128 && i <= 127) // must cache 
        { 
	    return IntegerCache.cache[i + offset];
	}
        return new Integer(i);
 }

So when creating an object using Integer.valueOf or directly assigning a value to an Integer within the range of -128 to 127 the same object will be returned. Therefore, consider the following example:

Integer i = 100;
Integer p = 100;
if (i == p)  System.out.println("i and p are the same.");
if (i != p)   System.out.println("i and p are different.");	
if(i.equals(p))  System.out.println("i and p contain the same value.");

The output is:

i and p are the same.
i and p contain the same value.

It is important to note that object i and p only equate to true because they are the same object, the comparison is not based on the value, it is based on object equality. If Integer i and p are outside the range of -128 or 127 the cache is not used, therefore new objects are created. When doing a comparison for value always use the “.equals” method. It is also important to note that instantiating an Integer does not create this caching. So consider the following example:

Integer i = new Integer (100);
Integer p = new Integer(100);
if(i==p) System.out.println(“i and p are the same object”);
if(i.equals(p)) System.out.println(“ i and p contain the same value”);

In this circumstance, the output is only:

i and p contain the same value

Remember that “==” is always used for object equality, it has not been overloaded for comparing unboxed values.

This behavior is documented in the Java Language Specification section 5.1.7. Quoting from there:

If the value p being boxed is true, false, a byte, a char in the range \u0000 to \u007f, or an int or short number between -128 and 127, then let r1 and r2 be the results of any two boxing conversions of p. It is always the case that r1 == r2.

The other wrapper classes (Byte, Short, Long, Character) also contain this caching mechanism. The Byte, Short and Long all contain the same caching principle to the Integer object. The Character class caches from 0 to 127. The negative cache is not created for the Character wrapper as these values do not represent a corresponding character. There is no caching for the Float object.

BigDecimal also uses caching but uses a different mechanism. While the other objects contain a inner class to deal with caching this is not true for BigDecimal, the caching is pre-defined in a static array and only covers 11 numbers, 0 to 10:

// Cache of common small BigDecimal values.
private static final BigDecimal zeroThroughTen[] = {
new BigDecimal(BigInteger.ZERO,		0,  0),
new BigDecimal(BigInteger.ONE,		1,  0),
new BigDecimal(BigInteger.valueOf(2),	2,  0),
new BigDecimal(BigInteger.valueOf(3),	3,  0),
new BigDecimal(BigInteger.valueOf(4),	4,  0),
new BigDecimal(BigInteger.valueOf(5),	5,  0),
new BigDecimal(BigInteger.valueOf(6),	6,  0),
new BigDecimal(BigInteger.valueOf(7),	7,  0),
new BigDecimal(BigInteger.valueOf(8),	8,  0),
new BigDecimal(BigInteger.valueOf(9),	9,  0),
new BigDecimal(BigInteger.TEN,		10, 0),
};

As per Java Language Specification(JLS) the values discussed above are stored as immutable wrapper objects. This caching has been created because it is assumed these values / objects are used more frequently.

Incrementing Values

Be careful of the post-increment operator:

 int x = 5;
 x = x++;
 System.out.println( x );

The output is:

5

Remember that the assignment completes before the increment, hence post-increment. Using the pre-increment will update the value before the assignment. For example:

int x = 5;
x = ++x;
System.out.println( x );

The output is:

6

Garbage Collection

Overriding “finalize()” will allow you to define you own code for what is potentially the same concept as a destructor. There are a couple of important points to remember:

  • “finalize()” will only ever by called once (at most) by the Garbage Collector.
  • It is never a guarantee that “finalize()” will be called i.e. that an object will be garbage collected.
  • By overriding “finalize()” you can prevent an object from ever being deleted. For example, the object passes a reference of itself to another object.
  • Garbage collection behaviour differs between JVMs.

Boolean Assignment

Everyone appreciates the difference between “==” and “=” in Java. However, typos and mistakes are made, and often the compiler will catch them. However, consider the following:

boolean theTruth = false;
if (theTruth = true)
{
 System.out.println("theTruth is true");
}
else
{
 System.out.println("theTruth is false;");
}

The result of any assignment expression is the value of the variable following the assignment. Therefore, the above will always result in “theTruth is true”. This only applies to booleans, so for example the following will not compile and would therefore be caught by the compiler:

int i = 1;
if(i=0) {}

As “i” is and integer the comparison would evaluate to (i=0) as 0 is the result of the assignment. A boolean would be expected, due the “if” statement.

Conditions

Be on the look out for any nested “else if”. Consider the following code example:

int x = 3;
if (x==5) {}
else if (x<9)
{
  System.out.println("x is less than 9");
}
else if (x<6)
{
  System.out.println("x is less than 6");
}
else
{
  System.out.println("else");
}

Produces the output:

x is less then 9

So even though the second “else if” would equate to “true” it is never reached. This is because once an “else if” succeeds the remaining conditions will be not be processed.

pretty-print JSON using JavaScript

var str = JSON.stringify(obj, null, 2); // spacing level = 2

If you need syntax highlighting, you might use some regex magic like so:

function syntaxHighlight(json) {
    if (typeof json != 'string') {
         json = JSON.stringify(json, undefined, 2);
    }
    json = json.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
    return json.replace(/("(\\u[a-zA-Z0-9]{4}|\\[^u]|[^\\"])*"(\s*:)?|\b(true|false|null)\b|-?\d+(?:\.\d*)?(?:[eE][+\-]?\d+)?)/g, function (match) {
        var cls = 'number';
        if (/^"/.test(match)) {
            if (/:$/.test(match)) {
                cls = 'key';
            } else {
                cls = 'string';
            }
        } else if (/true|false/.test(match)) {
            cls = 'boolean';
        } else if (/null/.test(match)) {
            cls = 'null';
        }
        return '<span class="' + cls + '">' + match + '</span>';
    });
}

See in action here: jsfiddle

检索分词相关

敏感词处理器,支持返回敏感词,高亮敏感词,替换敏感词等操作
sensitive-word-filter

简单的标题检索,核心思想就是”内存hash + ID list”(倒排索引),先分词,再根据hash(分词)进而查找匹配,然后聚合匹配结果得到检索列表。
title-search