在Scala中case class

 1、初始化的时候可以不用new,当然你也可以加上,普通类一定需要加new;

 2、toString的实现更漂亮;

 3、默认实现了equals 和hashCode;

 4、默认是可以序列化的,也就是实现了Serializable ;

 5、自动从scala.Product中继承一些函数;

 6、case class构造函数的参数是public级别的,我们可以直接访问;

 7、支持模式匹配;
  其实感觉case class最重要的特性应该就是支持模式匹配。这也是我们定义case class的唯一理由,难怪Scala官方也说:It makes only sense to define case classes if pattern matching is used to decompose data structures.。

How do I find all files containing specific text on Linux?

Do the following:

grep -rnw '/path/to/somewhere/' -e 'pattern'
  • -r or -R is recursive,
  • -n is line number, and
  • -w stands for match the whole word.
  • -l (lower-case L) can be added to just give the file name of matching files.

Along with these, --exclude, --include, --exclude-dir flags could be used for efficient searching:

  • This will only search through those files which have .c or .h extensions:
    grep --include=\*.{c,h} -rnw '/path/to/somewhere/' -e "pattern"
    
  • This will exclude searching all the files ending with .o extension:
    grep --exclude=*.o -rnw '/path/to/somewhere/' -e "pattern"
    
  • For directories it’s possible to exclude a particular directory(ies) through --exclude-dirparameter. For example, this will exclude the dirs dir1/, dir2/ and all of them matching *.dst/:
    grep --exclude-dir={dir1,dir2,*.dst} -rnw '/path/to/somewhere/' -e "pattern"
    

This works very well for me, to achieve almost the same purpose like yours.

For more options check man grep.

Generate 16-digit unique code (like product serial)

static final private String ALPHABET = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
final private Random rng = new SecureRandom();    

char randomChar(){
    return ALPHABET.charAt(rng.nextInt(ALPHABET.length()));
}

String randomUUID(int length, int spacing, char spacerChar){
    StringBuilder sb = new StringBuilder();
    int spacer = 0;
    while(length > 0){
        if(spacer == spacing){
            sb.append(spacerChar);
            spacer = 0;
        }
        length--;
        spacer++;
        sb.append(randomChar());
    }
    return sb;
}

使用webmagic构建一个分布式的爬虫

public class RedisScheduler implements Scheduler{

    private JedisPool pool;

    private static final String QUEUE_PREFIX = "queue_";

    private static final String SET_PREFIX = "set_";

    public RedisScheduler(String host){
        pool = new JedisPool(new JedisPoolConfig(), host);
    }

    @Override
    public void push(Request request, Task task) {
        Jedis jedis = pool.getResource();
          //使用SortedSet进行url去重
        if (jedis.zrank(SET_PREFIX+task.getUUID(),request.getUrl())==null){
            //使用List保存队列
            jedis.rpush(QUEUE_PREFIX+task.getUUID(),request.getUrl());
            jedis.zadd(SET_PREFIX+task.getUUID(),System.currentTimeMillis(),request.getUrl());
        }
    }

    @Override
    public Request poll(Task task) {
        Jedis jedis = pool.getResource();
        String url = jedis.lpop(QUEUE_PREFIX+task.getUUID());
            if (url==null) {
                return null;
            }
        return new Request(url);
    }
}