This text provides a review of state-of-the-art techniques as well as describing several ideas and algorithms for semantic object extraction from image sequences. Semantic object extraction is an essential element in content-based multimedia services, such as the newly developed MPEG4 and MPEG7 standards. An interactive system called SIVOG (Smart Interactive Video Object Generation) is presented, which converts user’s semantic input into a form that can be conveniently integrated with low-level video processing. Thus, high-level semantic information and low-level video features are integrated seamlessly into a smart segmentation system. A region and temporal adaptive algorithm was further proposed to improve the efficiency of the SIVOG system so that it is feasible to achieve nearly real-time video object segmentation with robust and accurate performances. Also included is an examination of the shape coding problem and the object segmentation problem simultaneously.