Hello world!

Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!

Posted in Uncategorized | 1 Comment

Call for Registration: ACM Multimedia 2010 Tutorials

ACM Multimedia 2010 Tutorial Program – Call For Registration 

ACM Multimedia 2010 tutorial program will provide in-depth learning and discussion opportunities on state-of-the-art multimedia research and development.  All tutorials will be held on the first day (Oct. 25, 2010) of the main conference. This year we are offering 14 tutorials that cover current and emerging topics from the four major tracks of the conference. The detailed information of all the tutorials can be found at: http://www.acmmm10.org/program/tutorials/. You can register for any of the tutorials through the main conference registration site at http://www.acmmm10.org/attendees/registration.

Posted in Research | Leave a comment

Interactive Image Search: Search by Color Sketch – Just Released in Microsoft Bing Image Search

On June 3, Microsoft released a new version of its Bing search engine. Among the new functionalities, an interactive image search interface provides a totally new capacity of searching images on the Web. This article will introduce this feature with a couple of interesting examples.

The new feature is based on a research effort from Microsoft Research Asia (MSRA). Last year, the two major contributors for this technique from MSRA, Xian-Sheng Hua and Jingdong Wang, has reported this technology using a number of screenshots of a prototype in MSRA Blog (Chinese version). Now this interesting technique is in Bing image search that everyone can try.

Originally the feature is called “search by color map” in the prototype. But now it is called “Specific color” and “Filter using sketch” in Bing image search. We can call it “image search by color sketch”.

In this article, we will first introduce the steps of using this feature, which has three functionalities, and then present more details and tips of using this feature. And last we will talk a little bit about the technology behind.

Table of Content

How to Use Search by Color Sketch

More Examples

Grass 
Flower 
Flag 
Search people (“Lady Gaga”) 
Search car (“BMW”)

About the Underlying Technologies

 

How to Use Search by Color Sketch

1. Visit http://www.bing.com and click “Images”, or directly visit: http://www.bing.com/images or http://image.bing.com;

2. Check the top-right corner of the web page and change the region of the service to “United States” if currently it is not. Currently this feature is only enabled in US region, but can be used worldwide as long as you changed the region to “United States”;

3. Type a query in the search box and press search button. Now we use “apple” as an example.

4. Click “COLOR” on the left panel (if it is not expanded), then you will see a menu “Specific color”. This is the entrance of the new feature. Currently it is hidden and hope it will be placed somewhere easier to be found (Figure 1).

(a) The whole page

(b) Enlarged: the bottom-right corner of the page

Figure 1: Open the entrance of the new feature.

5. After clicking, you get a panel at bottom-left part of the page (Figure 1), where you can try the first functionality: search by dominant color. That means, you can get “flower” related images but the dominant color of the images would be the one you clicked. For example, if you click the small blue block, you will get results like Figure 2 (mostly blue flowers or flowers’ color close to blue) (Figure 2)

Figure 2: Blue flowers (“zoom small” was clicked so much more images can be shown in one page, which can be set on top-right of the image area)

6. The second and third functionality needs open the drawing panel by checking “Filter using sketch” in the color panel. Then you can choose colors and draw the color sketch of the desired image (Figure 3). Please note this is not for drawing contour of a cat, dog, house or other objects, but only the color “map” of desired images. For example, we may want to find purple flowers with green leaf – that’s what Figure 3 is showing. After drawing the color sketch, press “Apply” to get new search results. We can call this “search by drawing” or “search by color sketch”.

Figure 3: Purple flowers with green leaves.

7. The third functionality could be called “search by modified example”. That means, instead of drawing on the empty panel, we can drag-n-drop an image from the search results to the panel. We can directly click “Apply” to search similar images (Figure 4), or we can draw something to change the example before search. For example, we use the eraser to ease the green leaves and then draw some yellow blocks, which indicates that I’d like to see some yellow followers or parts besides the purple ones (Figure 5).

Figure 4: Search by an example (drag-n-drop an image from the search results).

Figure 5: Search by modified example (purple flowers but with something yellow). The first image in the results will remain the same as the originally selected image no matter how you modified it. This is by design, so people will always know which image he or she has chosen.

There are hyperlinks embedded in the example pictures above (and also those in next section). You can click it to see what it looks like in Bing image search. However, the results may be slightly different (or even much different if you read this article long after June 2010) with the pictures showing here as Bing’s image index is keeping updating. If the links are invalid, you may try it manually using the same term and similar drawing, or any other drawings you like.

Below figure shows the functions of the button in the panel (Figure 6).

Figure 6: Functions of the buttons.

More Examples

We will show you a number of examples, based on which you will learn how to well express you intent through drawing color sketch.

Grass

Type “grass” in Bing image search you will get something like Figure 7.

Figure 7: Original results of “grass” in Bing image search.

If want more sky on the top and less grass at the bottom, you can draw something to express this intent and get something like Figure 8.

Figure 8: “Grass” images with more sky on top and less grass at bottom.

Or you want less sky on top and more grass at bottom (Figure 9):

Figure 9: “Grass” images with little sky and lots of grass.

Sometimes you may want to find “orange grass” (Figure 10), or you find one of the orange grass looks more like what you want (drag it to the panel as an example) (figure 11).

Figure 10: Orange grass.

Figure 11: Search by example “grass”

Sometimes, you may want to get some grass images with clean background, so it will be easier to use it. This can be achieved by set the background to white or black or any other possible background color (Figure 12).

Figure 12: Grass images with clean/white background

Another example of searching by example: grass with orange sky (Figure 13), and modified example (similar grass with blue sky) (Figure 14).

Figure 13: Search by example (grass with orange sky area).

Figure 14: Search by modified example (similar grass but with blue sky). Again, the first image in the results will remain the same as the originally selected image no matter how you modified it. This is by design, so people will always know which image he or she has chosen.

Flower

Original results of searching “flower” in Bing image search (Figure 15). Blue flowers or some flower related examples have been shown in Figure 2, 3, 4 and 5. Here are more examples: purple flower (Figure 16), yellow flower (Figure 17), yellow flower with dark background (Figure 18), search by example (an yellow flower with dark background) (Figure 19), and search by modified example (yellow flower with green background)(Figure 20).

Figure 15: Original results of “flower” in Bing image search.

Figure 16: Purple flowers (by selecting the dominant color).

Figure 17: Yellow flowers (indicated by yellow blocks in the center area).

Figure 18: Search by drawing (yellow flowers with dark background, indicated by yellow blocks at center area and black blocks at the border).

Figure 19: Search by example (yellow flower with dark background).

Figure 20: Search by modified example (change the background to green, indicating leaves or grass).

Flag

When you saw a flag of a country when watching world cup, but you don’t know what the country of the flag, you may use Bing image search to find it out. Below shows several results (Figure 21 to Figure 27) – after drawing the flag roughly, you get the real flag images. Mouse-over or click the desired result, you will know the country of the flag.

Figure 21: China flag.

Figure 22: India flag (and other similar flags, such as Miami city flag and Niger flag)

Figure 23: Niger flag (and other similar ones).

Figure 24: Ghana flag.

Figure 25: Mauritius flag (search by example).

Figure 26: Gambia flag (search by modified example).

Figure 27: Danmark flag and similar flags (search by drawing).

My son (2.5 years old) is able to recognize more than 80 flags of different countries, but I can only recognize 10+. Now with Bing’s “search by color sketch”, I can compete with him now, thought the speed is much slower than him :)

Search people (“Lady Gaga”)

“Lady Gaga” is very popular recently. Figure 28 shows the initial results in Bing. Figure 29, 30, 31, and 32 shows the images of Lady Gata walking on red carpet and wearing in different colors.

Figure 28: Original results of “Lady Gaga” from Bing image search.

Figure 29: Lady Gaga walking on red carpet (indicated by a red line at the bottom).

Figure 30: Lady Gaga walking on red carpet and wearing in black (indicated by a red line at the bottom and some black blocks near the center.

Figure 31: The same as above but wearing in white (sometimes you need to adjust the size and/or position of the color blocks to get better results).

Figure 32: Walking on red carpet while background is white.

Figure 33: Search by an example (wearing in black and blue).

Figure 34: Dark background.

Figure 35: Dark background with larger face area.

Figure 36: Search by another example with dark background.

Search car (“BMW”)

Figure 37 to Figure 43 show a couple examples for search color of different colors and different background.


Figure 37: Original results of “BMW” of Bing.

Figure 38: Blue BMW.

Figure 39: Red BMW.

Figure 40: Red BMW on lawn.

Figure 41: Red BWM on lawn with blue sky.

Figure 42: Search by modified example. Originally the car is black (i.e., the first image in the result list). Now it was changed to red – we got red cars in similar background.

Figure 43: The same as the above example, but we change the color of the car to orange.

About the Underlying Technologies

This is not re-ranking, not conventional “query by sketch”, not just user-interface. It should be the first web-scale “image search by drawing” system. When drawing is provided, a new query based on the text query and drawing will be issued to search new images from the entire Bing index.

……

(To be continued – some high-level introduction of the technology and the differences compared with existing work, as well as some tips of using the feature.)

By Xian-Sheng Hua, Beijing, June 2010

Posted in Research | 8 Comments

Microsoft Academic Search (paper search)

http://academic.research.microsoft.com/

 Try it. Still in beta.

 (I was asked to give a try by some people working on it :))

Pros:

Good UI. Support author search/rank list, conference search/rank list, journal search/rank list, and some other cool features (try it J)

For example, top ranked authors in multimedia area:

http://academic.research.microsoft.com/CSDirectory/Author_category_13.htm  (hope it was done automatically)

And it also support author profile page, for example: http://academic.research.microsoft.com/Author/951346.aspx 

However, the big problem is it cannot differentiate different people with the same name either. But seems there is no a search engine can solve this problem currently.

 

Cons:

Dataset is not complete enough. Missing papers, citations, and indexes (such as g-index and h-index) are much blow the correct ones.

For the nice feature mentioned above (author profile), the downside is that it also provides a photo of the author, however, the listed papers may come from different people with the same name. The person whose papers are listed but the photo is not his/her may get angry about it :)

 

I believe they are improving, very quickly.I heard they are updating data every week and updating features every two months.

Posted in Research | 1 Comment

[转]青菜的味道

早上上班路上听了一个广播故事,颇有同感。

【播文】
   有一个农民兄弟这样感慨:我们好不容易吃上肉,你们城里人又改吃青菜了。国外也有一种流行说法:穷人吃肉,富人吃青菜。居城多年,聚餐时有一个细节让我记忆深刻──在合上点菜单的时候,宾主间总会有人补充一句﹕“来个青菜吧,最好是带叶子的那种。”

  青菜,神奇的青菜,有时声势浩大地在贫富和城乡之间划一道分界线,有时仅仅是一桌菜品中微不足道的小配角。不管贫富,也不论城乡,过日子终归是离不开青菜的,念念青菜生活味。

  有一次,朋友问我:“你知道青菜的味道吗﹖”第一反应竟是茫然,缓过神来,怀疑他在给我开类似脑筋急转弯的玩笑。我敛起笑容,煞是认真地戏谑道﹕“青菜的味道?不就是青菜味道嘛!”

  朋友却一脸认真地说﹕“我刚刚从成都回来。川菜久负盛名,那些日子我算是真正领略到了。印象最深的不是大鱼大肉,麻辣水煮,而是一家并不起眼的小馆子里的青菜。那家餐馆名字也很有意思,叫青菜人家,土得掉渣,像个裹着头巾羞涩的村姑。进得店来,迎面一张屏风写了一句──你知道青菜的味道吗﹖”

  我等不及,反问道﹕“那你知道青菜的味道吗﹖”朋友说:“从那家店里出来,就知道了。青菜人家有一道水煮青菜,不加任何着料,在滚水里过一遍,就端上桌。吃起来,才是真正的青菜味道。那味道其实很平淡,有些涩口,带点草香,下肚后有回甘,就像割草机修剪过草坪之后,空气里流淌的那味儿了。”

  我说:“不稀奇,国外有人吃青菜都不用在开水过一遍,生吃。那才是叫原味呢。”

  朋友说:“你说得不对,他们吃生的青菜,都要蘸五花八门的酱呢。”

  蓦然间,我想起小时候吃的青菜来。那时,家里做饭用大木甑蒸,饭熟了,母亲会将洗净的青菜──印象中空心菜占多数,放进蒸饭的滚水一焯,洒上一小勺盐,就直接下饭。儿时吃的青菜,没有酱醋等诸般纠缠,没有经烈火锅油炙烤,不走偏,不失真,吃进嘴里,是青菜的原味。只有吃过这样的青菜,才有资格回答“青菜是什么味道”这一并不复杂的问题。可是,红尘中的你我,有几人吃过这样清淡寡味的青菜呢?

  多年来,我们盘中和嘴里的青菜,都被旺火开发过,被油盐酱醋浸润过,其味都被各种杂味抢了风头,遮盖了去,真味反而模糊起来,记不清晰了。

  青菜味道,恰如人生万般滋味,缠绕在味蕾上的是理不清道不明的枝枝蔓蔓,原味和真味,往何处寻觅呢?

  人生之初,我们如吃寡淡的原味青菜。当然,谁也不愿意长久地这样淡下去,所以,不断地树立自己远大的人生目标,苦苦奋斗,孜孜以求。

  等梦想实现了,就如我们吃那被各种调料包围的青菜,原味道于种种干扰中,模糊了,失偏了。正所谓人生百味。老之将至,万事放下,心头了无挂碍,经过百般滋味的历练,终究回归至原味,就像我那朋友从“青菜人家”那里所尝到的原味青菜,就像我儿时吃的那滚水里焯一遍的青菜。

  青菜的味道,从另一角度看,是微版的百味人生。

  人们总在努力追求着,苦苦追寻着,却在不断追求中,丢失了自己最初的目标。等到我们想起那时本真的愿望,回味当初的向往,才发现事过境迁人渐老。

  走在人生道路上,走着走着,不知不觉把自己走丢了,迷失在诱惑、欲望、惊喜、烦恼、痛苦和悔恨等杂陈的人生滋味中。老子曰:“大音希声,大象无形。”我想补充一句:大味至淡。“你知道青菜的味道吗?”一语惊心,不由地在内心自问──你知道成长的味道吗?你知道读书的味道吗?你知道爱的味道吗?你知道……

  味道的迷失,似乎存在于人生的每一时,每一处,处在迷失的当头,各种本真的原味对我们来说,有时竟会是那么奢侈。

  (摘自香港《大公报》 作者:志宏)

【简评】

与《菜根谭》上的这句可互相印证:“甘辛膿肥非真味,真味只是淡,神奇卓異非至人,至人只是常。”不要忘了自己最初的目标,不要迷失在为达到目标而选择路径上 – 那还可能不是最好的路径。

《老子》第四十一章原文:“上士闻道,勤而行之;中士闻道,若存若亡;下士闻道,大笑之。不笑不足以为道。故建言有之: 明道若昧;进道若退;夷道若纇;上德若谷;大白若辱;广德若不足;建德若偷;质真若渝;大方无隅;大器晚成;大音希声;大象无形;道隐无名。夫唯道,善贷且成。”

Posted in Uncategorized | 3 Comments

“查颜观色”

最近为MSRA博客写的一篇短文,介绍基于颜色的图像搜索:
 
 
欢迎大家去踩踩
Posted in Research | 4 Comments

做学问当效季老

早起上班路上听广播,收音机里讲了刚刚辞世的季羡林老先生关于“国学大师”和“国宝”的几段文字,感触颇深 — 做学问当效季老:
 
辞“国学大师”:
 
    现在在某些比较正式的文件中,在我头顶上也出现“国学大师”这一灿烂辉煌的光环。这并非无中生有,其中有一段历史渊源。

  约摸十几二十年前,中国的改革开放大见成效,经济飞速发展。文化建设方面也相应地活跃起来。有一次在还没有改建的北京大学大讲堂里开了一个什么会,专门向同学们谈国学。当时主席台上共坐着五位教授,每个人都讲上一通。我是被排在第一位的,说了些什么话,现在已忘得干干净净。一位资深记者是北大校友,在报上写了一篇长文《国学热悄悄在燕园兴起》。从此以后,其中四位教授,包括我在内,就被称为“国学大师”。他们三位的国学基础都比我强得多。他们对这一顶桂冠的想法如何,我不清楚。我自己被戴上了这一顶桂冠,却是浑身起鸡皮疙瘩。

  说到国学基础,我从小学起就读经书、古文、诗词。对一些重要的经典著作有所涉猎。但是我对哪一部古典,哪一个作家都没有下过死功夫,因为我从来没想成为一个国学家。后来专治其他的学术,浸淫其中,乐不可支。除了尚能背诵几百首诗词和几十篇古文外;除了尚能在最大的宏观上谈一些与国学有关的自谓是大而有当的问题比如天人合一外,自己的国学知识并没有增加。环顾左右,朋友中国学基础胜于自己者,大有人在。在这样的情况下,我竟独占“国学大师”的尊号,岂不折煞老身(借用京剧女角词)!我连“国学小师”都不够,遑论“大师”!

  为此,我在这里昭告天下:请从我头顶上把“国学大师”的桂冠摘下来。

 
     (见季羡林《病榻杂记》)。
 
辞“学界泰斗”
 
  这要分两层来讲:一个是教育界,一个是人文社会科学界。

  先要弄清楚什么叫“泰斗”。泰者,泰山也;斗者,北斗也。两者都被认为是至高无上的东西。

  光谈教育界。我一生做教书匠,爬格子。在国外教书10年,在国内57年。人们常说:“没有功劳,也有苦劳。”特别是在过去几十年中,天天运动,花样翻新,总的目的就是让你不得安闲,神经时时刻刻都处在万分紧张的情况中。在这样的情况下,我一直担任行政工作,想要做出什么成绩,岂不戛戛乎难矣哉!我这个“泰斗”从哪里讲起呢?

  在人文社会科学的研究中,说我做出了极大的成绩,那不是事实。说我一点成绩都没有,那也不符合实际情况。这样的人,滔滔者天下皆是也。但是,现在却偏偏把我“打”成泰斗。我这个泰斗又从哪里讲起呢?

  为此,我在这里昭告天下:请从我头顶上把“学界(术)泰斗”的桂冠摘下来。

 
辞“国宝”
 
  在中国,一提到“国宝”,人们一定会立刻想到人见人爱憨态可掬的大熊猫。这种动物数量极少,而且只有中国有,称之为“国宝”,它是当之无愧的。可是,大约在八九十来年前,在一次会议上,北京市的一位领导突然称我为“国宝”,我极为惊愕。到了今天,我所到之处,“国宝”之声洋洋乎盈耳矣。我实在是大惑不解。当然,“国宝”这一顶桂冠并没有为我一人所垄断。其他几位书画名家也有此称号。

  我浮想联翩,想探寻一下起名的来源。是不是因为中国只有一个季羡林,所以他就成为“宝”。但是,中国的赵一钱二孙三李四等等,等等,也都只有一个,难道中国能有13亿“国宝”吗?

  这种事情,痴想无益,也完全没有必要。我来一个急刹车。

  为此,我在这里昭告天下:请从我头顶上把“国宝”的桂冠摘下来。

 
媒体赞季老:虚怀若谷 对虚荣社会无声抗议 (原文
 
  香港大公报文章指出,每每说到季羡林老先生,国人无不敬重有加,这不仅因为他渊博的知识,他不但精通几十种外文,对一些已经面临绝迹的文字也颇有研究,尤其对中世纪的印欧语言的研究达到了相当的水平,同时,他平实的为人更是令人称道。近几天中央电视台播放的“2006感动中国”的节目中,有一组对季老先生的采访镜头令人难忘。有一个掏粪工是一个业余画家,他出了自己的画集之后想请名家作序,找了几个颇有点名气的人作序时,人家都以种种理由予以回绝。而我们的季老先生却没有嫌弃这位掏粪工,慷慨应允作序。一滴水可以见太阳,从这件小事中人们可以领略季老先生的人生风范。

  文章指出,国学大师、学界泰斗、国宝,这些含金量极高令人羡慕的字眼,是多少人梦寐以求的,而在季老那里却十分地看轻。可以说,季老先生以自己的实际行动给那些追捧虚荣的人上了很好的一课。三项诱人的桂冠就这样被我们的季老先生“轻易”地自己给自己摘去了,这在很多人看来是多么地不可思议!要知道,那是很多人终身奋斗的目标啊!这也充分反映了季老先生“视功名如过眼烟云”的坦荡胸怀。

  季羡林的“请辞”大有深意,这样的“请辞”是对贪慕虚荣社会的一种无声的抗议!给我们的每个社会中人上了很好的一课。论才学也好,论人品也好,季老都当之无愧,可他老人家却并没有如此地“看重”自己,而是把自己当成一个普通的百姓,这样的胸怀正是我们这个社会所缺少的一种精神资源。面对物欲横流的社会,很多人开始丧失自己的道德操守,在学术界更是上演了一幕幕肮脏的“闹剧”,为了获得高级职称,不惜造假,为了显示学术成果,不惜剽窃别人的论文,所有这些都严重地败坏着学界风气。不但如此,在其它领域,贪慕虚荣,崇尚奢华的风气日甚一日,如果我们的社会被这样一种虚假的气氛所笼罩,国家未来的前途令人堪忧。

  社会文明和进步的重要标志是崇尚实干,而不是欺上瞒下,欺世盗名。这方面,季老先生给国人树立了一个很好的榜样。愿世人能从季老先生的“请辞”中得到教益。

  香港文汇报就此还采访了季羡林先生。

  记 者:现在社会上和学术界对您有些微词。
  季羡林:有微词好。
  记 者:您为什么要摘掉自己头上的光环?像是“国宝”、“泰斗”、“大师”这些称号?
  季羡林:我觉得自己不够格,所以想摘掉。
  记 者:您觉得谁够格?
  季羡林:有很多。(迟疑了一下)鲁迅是一个。在那个时代,鲁迅的骨头是最硬的。
  记 者:您对这个民族做出的贡献,是有目共睹的。
  季羡林:我在国外多年,回来后做了一点事,给我的荣誉太高太多,我担受不起。我的人生轨迹是直线向上的,很简单。

Posted in Research | Leave a comment