You need to import the Joomla library to be able to use it's API, like so:
<?php
define('_JEXEC', 1);
define('JPATH_BASE', realpath(dirname(__FILE__) . '/../../'));
require_once JPATH_BASE . '/includes/defines.php';
require_once JPATH_BASE . '/includes/framework.php';
$mainframe = JFactory::getApplication('site');
?>
You may need to change the path on line 2 of the code above depending on where Joomla is located in relation to your custom PHP file.