Feature - Importing YouTube videos into Drupal with Feeds and Json

There are several modules already on Drupal that let you import YouTube videos in various ways, but in my experience none of them quite do what I want and have various dependencies that wind me up a bit.

Feeds_youtube  is the obvious choice for most and does a pretty good job but I wanted to grab more of the available data available on the YouTube data API. Here is a list of current supported items from the feeds_youtube module. If this is enough for you then don’t read any further:

• Video ID
• Video title
• Author
• Updated on (Datetime)
• Published on (Datetime)
• Description
• Thumbnail
• Category (Can be imported seamlessly with Taxonomy module)
• Tags (Can be imported seamlessly with Taxonomy module)
• Watch page URL (Can be imported seamlessly with Embedded Media Field module)
• Duration (Formatted hh:mm:ss)
• Favorite count
• View count
• Rating
• GUID
• Updated on (Timestamp)
• Published on (Timestamp)
• Duration (Seconds)

But lets take a look at an example from my favorites feed in json-c format from v2 of the API as semi-documented here:

{
        "id": "vjVQa1PpcFNJO_-HMJBDHCWFxzBFULppLPxwLFXfXwA",
        "created": "2011-11-18T22:44:47.000Z",
        "video": {
          "id": "QH2-TGUlwu4",
          "uploaded": "2011-04-06T03:21:59.000Z",
          "updated": "2011-11-24T17:13:17.000Z",
          "uploader": "saraj00n",
          "category": "Comedy",
          "title": "Nyan Cat [original]",
          "description": "etc etc etc",
          "tags": [
            "cat",
            "flying through space",
            "annoying",
            "amazing"
          ],
          "thumbnail": {
            "sqDefault": "http://i.ytimg.com/vi/QH2-TGUlwu4/default.jpg",
            "hqDefault": "http://i.ytimg.com/vi/QH2-TGUlwu4/hqdefault.jpg"
          },
          "player": {
            "default": "https://www.youtube.com/watch?v=QH2-TGUlwu4&feature=youtube_gdata_player",
            "mobile": "https://m.youtube.com/details?v=QH2-TGUlwu4"
          },
          "content": {
            "5": "https://www.youtube.com/v/QH2-TGUlwu4?version=3&f=user_favorites&app=youtube_gdata",
            "1": "rtsp://v2.cache8.c.youtube.com/CioLENy73wIaIQnuwiVlTL59QBMYDSANFEgGUg51c2VyX2Zhdm9yaXRlcww=/0/0/0/video.3gp",
            "6": "rtsp://v6.cache4.c.youtube.com/CioLENy73wIaIQnuwiVlTL59QBMYESARFEgGUg51c2VyX2Zhdm9yaXRlcww=/0/0/0/video.3gp"
          },
          "duration": 217,
          "rating": 4.779854,
          "likeCount": "494592",
          "ratingCount": 523398,
          "viewCount": 49237479,
          "favoriteCount": 283946,
          "commentCount": 486974,
          "accessControl": {
            "comment": "allowed",
            "commentVote": "denied",
            "videoRespond": "denied",
            "rate": "allowed",
            "embed": "allowed",
            "list": "allowed",
            "autoPlay": "allowed",
            "syndicate": "allowed"
          }
        }
      }

There is definitely some valuable information in there that is going to make your life a lot easier if you are importing in bulk such as restrictions, access controls, full stats, all player versions and so on.

So basically I came up with a nice clean solution that lets me import from any of the video feeds (user favorites/uploads, all the YouTube standard video feeds or a specific playlist) as supported and shown in this nifty tool here which you can also use for assisting you in getting the right url when building your feeds.

Rather than me walk you through a long tedious process of installing the right modules and setting up all the required fields I have packaged the whole set up as a feature that is attached to the bottom of this post.

1. I recommend doing this on a totally fresh install of Drupal 7. To make life simpler for yourself, make sure you have first downloaded the following modules to your module folder:

• ctools
• date
• features
• feeds
• feeds_jsonpath_parser
• feeds_tamper
• job_scheduler

Also, make sure to download the jsonpath-0.8.1.php file to the feeds_jsonpath_parser module folder (you might still get a small error after installing the feature in step 2 but as long as you have put the php file in the parsers folder it will go away by step 3).

2. Download the attached feature and install it as per this quick howto.

3. Go to create content on your site and add a new YouTube Importer. Put what you like as title and then put in a valid url (see here for details). Click save and you should now have a bunch of new YouTube Video nodes with ALL the information included!

Yes, its that easy. You can check out the results at http://tube.pvbw.com thanks to Gordon Ramsay.

The more observant of you will notice a couple of things:

1. I do not insert the video into any other modules such as video_filter, media_youtube or any more you might find on this list. Instead I embed the video directly using full html format as per YouTube’s player parameters. I do this by making a quick rewrite with feeds_tamper on the import and do it so that I can customise the player with whatever features I chose. In this feature I do autohide=1, showinfo=0, rel=0 and iv_load_policy=3 to make what I consider a more pleasant player. You can change this by editing the feeds_tamper on the player field to your liking. If you really want to, you can still import the file into media_youtube  using media_feeds module.
2. GUID in the node processor mapping settings of feeds is set to use the unique video ID. This will mean no duplicate videos are added to your site even if they are in multiple playlists/favorites etc. If you WANT the videos duplicated (perhaps you want to have separate playlist views and so on) then you should change the jsonpath parser settings for GUID to “id” instead of “video.id”.
3. The title of the new YouTube video nodes created will be the same as the YouTub video’s title. You can make this blank if you want to use your own title or auto_nodetitle for example.
4. Nearly all fields are set as text but can be changed to what you like just delete and add new with same name if there is a different format such as taxonomy you want to take advantage of.
5. Duration of seconds is not really supported by date so I used integer.
6. Aspect ratio is kind of useless but there anyway.
7. Status value and reason fields have all the API options. Most of them you will never see especially in a feed, but I put them in for completeness.

Please feel free to make any suggestions or criticisms.

UPDATE: I have created a drupal sandbox project for this for those that may be interested in cloning from git to try out.

UPDATE 2011-12-21: Its now a module available at http://drupal.org/project/youtube_importer_feature