junyanz · HackerTrans

It's possible. But 3D data (3D models, videos) are much more difficult to model via a deep neural net. While most of the researchers focus on modeling 2d images in recent years, there are a few work on 3D. For example, here is a project on modeling 3D objects like chairs and tables. https://arxiv.org/abs/1411.5928

junyanz

10 anni fadiscuss

We are working on something similar to your idea. We generate sketch images from real images automatically and train a model on the sketch images. So ideally, if a user draw the left wheel of the bicycle, the system will produce the entire bicycle sketch. We will release this 'sketch' feature in a few days and hope it will help a user better sketch object.

As you said, one can also apply other filters like Prisma.

junyanz

10 anni fadiscuss

I actually did two internship there. The Seattle lab did ship many new features like content-aware fill, shake reduction introduced in Photoshop CC 2015. The researchers there also has lots of freedom to explore different directions not directly related to the products, and it turns out that many of them will become a new feature of products within a few years.

junyanz

10 anni fadiscuss

https://github.com/psal/anonymouth Nice logo:)

junyanz

10 anni fadiscuss

This is a brilliant idea. I guess it would be difficult for this work to accomplish as you need to train a neural net on tons of data (like 100k images, or millions), and we cannot find so many paintings with consistent style.

Work like Deep style transfer, or Prisma can try to transfer the style of one painting to an existing user photo. But you cannot use it as painting tool for creating new stuff.

junyanz

10 anni fadiscuss

Sure. The current generative models cannot produce good details, and the generated images are often low resolution (e,g. 64x64). In the paper, we tried to enhance the low res result by stealing the high res details from the original photo. But in general, there are not many things you can do.

On the other hand, in the recent years, we see dramatic improvement of image quality from these generative models. Overall I think this is a promising and exciting direction.

junyanz

10 anni fadiscuss

Yes! In 1948, Shannon proposed using a Markov chain to create a statistical model of the sequences of letters in a piece of English text and this model can be used to generate random text given some existing text. (http://www.cs.princeton.edu/courses/archive/spr05/cos126/ass...). Here is a GitHub implementation: https://github.com/jsvine/markovify Deep models like LSTM/RNN can probably produce better results.

junyanz

10 anni fadiscuss

You are absolutely right. I believe this deep learning technique is a fancy way for mixing many many images automatically given a user's guidance.

junyanz

10 anni fadiscuss

Thanks for sharing our work. Check out the full video: https://m.youtube.com/watch?v=9c4z6YsBGQ0

This work is a deep learning extension of our previous average image project: https://m.youtube.com/watch?v=1QgL_aPPCpM. See the New Yorker article for details: http://www.newyorker.com/tech/elements/out-of-many-one

I guess the deep learning might be a better way to blend millions of images for creating new visual content.

junyanz

10 anni fadiscuss

Thank you guys for the interest. The interactive image generation interface as well as trained models are available at https://github.com/junyanz/iGAN. It is still under active development and let me know if you find bugs or would like to suggest new features.

comments