Troplo/PlexKodiConnect
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Merge pull request #1120 from croneter/beta-version

Browse source 
Bump master
This commit is contained in:
croneter 2020-02-19 15:53:31 +01:00 committed by GitHub
commit 9c67283085
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
39 changed files with 1224 additions and 1093 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -1,5 +1,5 @@
[![stable version](https://img.shields.io/badge/stable_version-2.10.4-blue.svg?maxAge=60&style=flat) ](https://github.com/croneter/binary_repo/raw/master/stable/repository.plexkodiconnect/repository.plexkodiconnect-1.0.2.zip) [![stable version](https://img.shields.io/badge/stable_version-2.10.12-blue.svg?maxAge=60&style=flat) ](https://github.com/croneter/binary_repo/raw/master/stable/repository.plexkodiconnect/repository.plexkodiconnect-1.0.2.zip)
[![beta version](https://img.shields.io/badge/beta_version-2.10.4-red.svg?maxAge=60&style=flat) ](https://github.com/croneter/binary_repo/raw/master/beta/repository.plexkodiconnectbeta/repository.plexkodiconnectbeta-1.0.2.zip) [![beta version](https://img.shields.io/badge/beta_version-2.10.12-red.svg?maxAge=60&style=flat) ](https://github.com/croneter/binary_repo/raw/master/beta/repository.plexkodiconnectbeta/repository.plexkodiconnectbeta-1.0.2.zip)
[![Installation](https://img.shields.io/badge/wiki-installation-brightgreen.svg?maxAge=60&style=flat)](https://github.com/croneter/PlexKodiConnect/wiki/Installation) [![Installation](https://img.shields.io/badge/wiki-installation-brightgreen.svg?maxAge=60&style=flat)](https://github.com/croneter/PlexKodiConnect/wiki/Installation)
[![FAQ](https://img.shields.io/badge/wiki-FAQ-brightgreen.svg?maxAge=60&style=flat)](https://github.com/croneter/PlexKodiConnect/wiki/faq) [![FAQ](https://img.shields.io/badge/wiki-FAQ-brightgreen.svg?maxAge=60&style=flat)](https://github.com/croneter/PlexKodiConnect/wiki/faq)

41
addon.xml
View file

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<addon id="plugin.video.plexkodiconnect" name="PlexKodiConnect" version="2.10.4" provider-name="croneter"> <addon id="plugin.video.plexkodiconnect" name="PlexKodiConnect" version="2.10.12" provider-name="croneter">
<requires> <requires>
<import addon="xbmc.python" version="2.1.0"/> <import addon="xbmc.python" version="2.1.0"/>
<import addon="script.module.requests" version="2.9.1" /> <import addon="script.module.requests" version="2.9.1" />
@ -83,7 +83,44 @@
<summary lang="lt_LT">Natūralioji „Plex“ integracija į „Kodi“</summary> <summary lang="lt_LT">Natūralioji „Plex“ integracija į „Kodi“</summary>
<description lang="lt_LT">Prijunkite „Kodi“ prie „Plex Medija Serverio“. Šiame papildinyje daroma prielaida, kad valdote visus savo vaizdo įrašus naudodami „Plex“ (ir nė vieno su „Kodi“). Galite prarasti jau saugomus „Kodi“ vaizdo įrašų ir muzikos duomenų bazių duomenis (kadangi šis papildinys juos tiesiogiai pakeičia). Naudokite savo pačių rizika!</description> <description lang="lt_LT">Prijunkite „Kodi“ prie „Plex Medija Serverio“. Šiame papildinyje daroma prielaida, kad valdote visus savo vaizdo įrašus naudodami „Plex“ (ir nė vieno su „Kodi“). Galite prarasti jau saugomus „Kodi“ vaizdo įrašų ir muzikos duomenų bazių duomenis (kadangi šis papildinys juos tiesiogiai pakeičia). Naudokite savo pačių rizika!</description>
<disclaimer lang="lt_LT">Naudokite savo pačių rizika</disclaimer> <disclaimer lang="lt_LT">Naudokite savo pačių rizika</disclaimer>
<news>version 2.10.4: <news>version 2.10.12:
- versions 2.10.5-11 for everyone
version 2.10.11 (beta only):
- Fix yet another rare but annoying bug where PKC becomes unresponsive during sync
version 2.10.10 (beta only):
- Fix rare but annoying bug where PKC becomes unresponsive during sync
- Fix PKC background sync not working in some cases
version 2.10.9 (beta only):
- Other Kodi add-ons can now search for Plex items using plugin://plugin.video.plexkodiconnect?mode=search&amp;query=YOUR SEARCH STRING HERE
version 2.10.8 (beta only):
- Improve thread pool management to render PKC snappier
- Attempt to fix broken pipe error
- Fix DirectPaths when a video's folder name is identical to a video's filename (you will need to manually reset the Kodi database)
version 2.10.7 (beta only):
- Fix PKC not starting up on iOS
- Optimize the new sync process and fix some bugs that were introduced
- Fix PKC becoming unresponsive e.g. when switching the PMS
version 2.10.6 (beta only):
- Fix AttributeError if user enters an invalid pin code
- Fix OperationalError when starting with a fresh PKC installation
- Fix IndexError
version 2.10.5 (beta only):
- Rewire library sync to speed it up and fix sync getting stuck in rare cases
- Optimize threads by using events instead of a polling mechanism. Fixes PKC becoming unresponsive, e.g. when switching users
- Optimize adding values to Kodi databases by not using sqlite COALESCE command
- Fix OperationalError when resetting PKC
- Improve sync resiliance when certain items are not to be synced to Kodi or PKC skipped an item in the past
- Make sure bool is returned instead of an int
- Don't use WAL mode for sqlite connections, it is not making any difference
version 2.10.4:
- version 2.10.3 for everyone - version 2.10.3 for everyone
- Fix to correctly wipe Kodi databases - Fix to correctly wipe Kodi databases

37
changelog.txt
View file

@ -1,3 +1,40 @@
version 2.10.12:
- versions 2.10.5-11 for everyone
version 2.10.11 (beta only):
- Fix yet another rare but annoying bug where PKC becomes unresponsive during sync
version 2.10.10 (beta only):
- Fix rare but annoying bug where PKC becomes unresponsive during sync
- Fix PKC background sync not working in some cases
version 2.10.9 (beta only):
- Other Kodi add-ons can now search for Plex items using plugin://plugin.video.plexkodiconnect?mode=search&query=YOUR SEARCH STRING HERE
version 2.10.8 (beta only):
- Improve thread pool management to render PKC snappier
- Attempt to fix broken pipe error
- Fix DirectPaths when a video's folder name is identical to a video's filename (you will need to manually reset the Kodi database)
version 2.10.7 (beta only):
- Fix PKC not starting up on iOS
- Optimize the new sync process and fix some bugs that were introduced
- Fix PKC becoming unresponsive e.g. when switching the PMS
version 2.10.6 (beta only):
- Fix AttributeError if user enters an invalid pin code
- Fix OperationalError when starting with a fresh PKC installation
- Fix IndexError
version 2.10.5 (beta only):
- Rewire library sync to speed it up and fix sync getting stuck in rare cases
- Optimize threads by using events instead of a polling mechanism. Fixes PKC becoming unresponsive, e.g. when switching users
- Optimize adding values to Kodi databases by not using sqlite COALESCE command
- Fix OperationalError when resetting PKC
- Improve sync resiliance when certain items are not to be synced to Kodi or PKC skipped an item in the past
- Make sure bool is returned instead of an int
- Don't use WAL mode for sqlite connections, it is not making any difference
version 2.10.4: version 2.10.4:
- version 2.10.3 for everyone - version 2.10.3 for everyone
- Fix to correctly wipe Kodi databases - Fix to correctly wipe Kodi databases

6
default.py
View file

@ -50,7 +50,8 @@ class Main():
plex_type=params.get('plex_type'), plex_type=params.get('plex_type'),
section_id=params.get('section_id'), section_id=params.get('section_id'),
synched=params.get('synched') != 'false', synched=params.get('synched') != 'false',
prompt=params.get('prompt')) prompt=params.get('prompt'),
query=params.get('query'))
elif mode == 'show_section': elif mode == 'show_section':
entrypoint.show_section(params.get('section_index')) entrypoint.show_section(params.get('section_index'))
@ -66,7 +67,8 @@ class Main():
entrypoint.browse_plex(key='/hubs/search', entrypoint.browse_plex(key='/hubs/search',
args={'includeCollections': 1, args={'includeCollections': 1,
'includeExternalMedia': 1}, 'includeExternalMedia': 1},
prompt=utils.lang(137)) prompt=utils.lang(137),
query=params.get('query'))
elif mode == 'route_to_extras': elif mode == 'route_to_extras':
# Hack so we can store this path in the Kodi DB # Hack so we can store this path in the Kodi DB

28
resources/lib/app/application.py
View file

@ -54,17 +54,18 @@ class App(object):
@property @property
def is_playing(self): def is_playing(self):
return self.player.isPlaying() return self.player.isPlaying() == 1
@property @property
def is_playing_video(self): def is_playing_video(self):
return self.player.isPlayingVideo() return self.player.isPlayingVideo() == 1
def register_fanart_thread(self, thread): def register_fanart_thread(self, thread):
self.fanart_thread = thread self.fanart_thread = thread
self.threads.append(thread) self.threads.append(thread)
def deregister_fanart_thread(self, thread): def deregister_fanart_thread(self, thread):
self.fanart_thread.unblock_callers()
self.fanart_thread = None self.fanart_thread = None
self.threads.remove(thread) self.threads.remove(thread)
@ -85,6 +86,7 @@ class App(object):
self.threads.append(thread) self.threads.append(thread)
def deregister_caching_thread(self, thread): def deregister_caching_thread(self, thread):
self.caching_thread.unblock_callers()
self.caching_thread = None self.caching_thread = None
self.threads.remove(thread) self.threads.remove(thread)
@ -111,6 +113,7 @@ class App(object):
""" """
Sync thread has done it's work and is e.g. about to die Sync thread has done it's work and is e.g. about to die
""" """
thread.unblock_callers()
self.threads.remove(thread) self.threads.remove(thread)
def suspend_threads(self, block=True): def suspend_threads(self, block=True):
@ -124,19 +127,16 @@ class App(object):
if block: if block:
while True: while True:
for thread in self.threads: for thread in self.threads:
if not thread.suspend_reached: if not thread.is_suspended():
LOG.debug('Waiting for thread to suspend: %s', thread) LOG.debug('Waiting for thread to suspend: %s', thread)
# Send suspend signal again in case self.threads # Send suspend signal again in case self.threads
# changed # changed
thread.suspend() thread.suspend(block=True)
if self.monitor.waitForAbort(0.1):
return True
break
else: else:
break break
return xbmc.abortRequested return xbmc.abortRequested
def resume_threads(self, block=True): def resume_threads(self):
""" """
Resume all thread activity with or without blocking. Resume all thread activity with or without blocking.
Returns True only if PKC shutdown requested Returns True only if PKC shutdown requested
@ -144,16 +144,6 @@ class App(object):
LOG.debug('Resuming threads: %s', self.threads) LOG.debug('Resuming threads: %s', self.threads)
for thread in self.threads: for thread in self.threads:
thread.resume() thread.resume()
if block:
while True:
for thread in self.threads:
if thread.suspend_reached:
LOG.debug('Waiting for thread to resume: %s', thread)
if self.monitor.waitForAbort(0.1):
return True
break
else:
break
return xbmc.abortRequested return xbmc.abortRequested
def stop_threads(self, block=True): def stop_threads(self, block=True):
@ -163,7 +153,7 @@ class App(object):
""" """
LOG.debug('Killing threads: %s', self.threads) LOG.debug('Killing threads: %s', self.threads)
for thread in self.threads: for thread in self.threads:
thread.abort() thread.cancel()
if block: if block:
while self.threads: while self.threads:
LOG.debug('Waiting for threads to exit: %s', self.threads) LOG.debug('Waiting for threads to exit: %s', self.threads)

18
resources/lib/artwork.py
View file

@ -33,8 +33,8 @@ class ImageCachingThread(backgroundthread.KillableThread):
if not utils.settings('imageSyncDuringPlayback') == 'true': if not utils.settings('imageSyncDuringPlayback') == 'true':
self.suspend_points.append((app.APP, 'is_playing_video')) self.suspend_points.append((app.APP, 'is_playing_video'))
def isSuspended(self): def should_suspend(self):
return any(getattr(obj, txt) for obj, txt in self.suspend_points) return any(getattr(obj, attrib) for obj, attrib in self.suspend_points)
@staticmethod @staticmethod
def _url_generator(kind, kodi_type): def _url_generator(kind, kodi_type):
@ -73,18 +73,26 @@ class ImageCachingThread(backgroundthread.KillableThread):
app.APP.deregister_caching_thread(self) app.APP.deregister_caching_thread(self)
LOG.info("---===### Stopped ImageCachingThread ###===---") LOG.info("---===### Stopped ImageCachingThread ###===---")
def _run(self): def _loop(self):
kinds = [KodiVideoDB] kinds = [KodiVideoDB]
if app.SYNC.enable_music: if app.SYNC.enable_music:
kinds.append(KodiMusicDB) kinds.append(KodiMusicDB)
for kind in kinds: for kind in kinds:
for kodi_type in ('poster', 'fanart'): for kodi_type in ('poster', 'fanart'):
for url in self._url_generator(kind, kodi_type): for url in self._url_generator(kind, kodi_type):
if self.wait_while_suspended(): if self.should_suspend() or self.should_cancel():
return return False
cache_url(url) cache_url(url)
# Toggles Image caching completed to Yes # Toggles Image caching completed to Yes
utils.settings('plex_status_image_caching', value=utils.lang(107)) utils.settings('plex_status_image_caching', value=utils.lang(107))
return True
def _run(self):
while True:
if self._loop():
break
if self.wait_while_suspended():
break
def cache_url(url): def cache_url(url):

408
resources/lib/backgroundthread.py
View file

@ -2,142 +2,241 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from __future__ import absolute_import, division, unicode_literals from __future__ import absolute_import, division, unicode_literals
from logging import getLogger from logging import getLogger
from time import time as _time
import threading import threading
import Queue import Queue
import heapq import heapq
from collections import deque
import xbmc import xbmc
from . import utils, app from . import utils, app, variables as v
WORKER_COUNT = 3
LOG = getLogger('PLEX.threads') LOG = getLogger('PLEX.threads')
class KillableThread(threading.Thread): class KillableThread(threading.Thread):
'''A thread class that supports raising exception in the thread from
another thread.
'''
# def _get_my_tid(self):
# """determines this (self's) thread id
# CAREFUL : this function is executed in the context of the caller
# thread, to get the identity of the thread represented by this
# instance.
# """
# if not self.isAlive():
# raise threading.ThreadError("the thread is not active")
# return self.ident
# def _raiseExc(self, exctype):
# """Raises the given exception type in the context of this thread.
# If the thread is busy in a system call (time.sleep(),
# socket.accept(), ...), the exception is simply ignored.
# If you are sure that your exception should terminate the thread,
# one way to ensure that it works is:
# t = ThreadWithExc( ... )
# ...
# t.raiseExc( SomeException )
# while t.isAlive():
# time.sleep( 0.1 )
# t.raiseExc( SomeException )
# If the exception is to be caught by the thread, you need a way to
# check that your thread has caught it.
# CAREFUL : this function is executed in the context of the
# caller thread, to raise an excpetion in the context of the
# thread represented by this instance.
# """
# _async_raise(self._get_my_tid(), exctype)
def kill(self, force_and_wait=False):
pass
# try:
# self._raiseExc(KillThreadException)
# if force_and_wait:
# time.sleep(0.1)
# while self.isAlive():
# self._raiseExc(KillThreadException)
# time.sleep(0.1)
# except threading.ThreadError:
# pass
# def onKilled(self):
# pass
# def run(self):
# try:
# self._Thread__target(*self._Thread__args, **self._Thread__kwargs)
# except KillThreadException:
# self.onKilled()
def __init__(self, group=None, target=None, name=None, args=(), kwargs={}): def __init__(self, group=None, target=None, name=None, args=(), kwargs={}):
self._canceled = False self._canceled = False
# Set to True to set the thread to suspended
self._suspended = False self._suspended = False
# Thread will return True only if suspended state is reached self._is_not_suspended = threading.Event()
self.suspend_reached = False self._is_not_suspended.set()
self._suspension_reached = threading.Event()
self._is_not_asleep = threading.Event()
self._is_not_asleep.set()
self.suspension_timeout = None
super(KillableThread, self).__init__(group, target, name, args, kwargs) super(KillableThread, self).__init__(group, target, name, args, kwargs)
def isCanceled(self): def should_cancel(self):
""" """
Returns True if the thread is stopped Returns True if the thread should be stopped immediately
""" """
if self._canceled or xbmc.abortRequested: return self._canceled or app.APP.stop_pkc
return True
return False
def abort(self): def cancel(self):
""" """
Call to stop this thread Call from another thread to stop this current thread
""" """
self._canceled = True self._canceled = True
# Make sure thread is running in order to exit quickly
self._is_not_asleep.set()
self._is_not_suspended.set()
def suspend(self, block=False): def should_suspend(self):
""" """
Call to suspend this thread Returns True if the current thread should be suspended immediately
""" """
return self._suspended
def suspend(self, block=False, timeout=None):
"""
Call from another thread to suspend the current thread. Provide a
timeout [float] in seconds optionally. block=True will block the caller
until the thread-to-be-suspended is indeed suspended
Will wake a thread that is asleep!
"""
self.suspension_timeout = timeout
self._suspended = True self._suspended = True
self._is_not_suspended.clear()
# Make sure thread wakes up in order to suspend
self._is_not_asleep.set()
if block: if block:
while not self.suspend_reached: self._suspension_reached.wait()
LOG.debug('Waiting for thread to suspend: %s', self)
if app.APP.monitor.waitForAbort(0.1):
return
def resume(self): def resume(self):
""" """
Call to revive a suspended thread back to life Call from another thread to revive a suspended or asleep current thread
back to life
""" """
self._suspended = False self._suspended = False
self._is_not_asleep.set()
self._is_not_suspended.set()
def wait_while_suspended(self): def wait_while_suspended(self):
""" """
Blocks until thread is not suspended anymore or the thread should Blocks until thread is not suspended anymore or the thread should
exit. exit or for a period of self.suspension_timeout (set by the caller of
Returns True only if the thread should exit (=isCanceled()) suspend())
Returns the value of should_cancel()
""" """
while self.isSuspended(): self._suspension_reached.set()
try: self._is_not_suspended.wait(self.suspension_timeout)
self.suspend_reached = True self._suspension_reached.clear()
# Set in service.py return self.should_cancel()
if self.isCanceled():
# Abort was requested while waiting. We should exit
return True
if app.APP.monitor.waitForAbort(0.1):
return True
finally:
self.suspend_reached = False
return self.isCanceled()
def isSuspended(self): def is_suspended(self):
""" """
Returns True if the thread is suspended Check from another thread whether the current thread is suspended
""" """
return self._suspended return self._suspension_reached.is_set()
def sleep(self, timeout):
"""
Only call from the current thread in order to sleep for a period of
timeout [float, seconds]. Will unblock immediately if thread should
cancel (should_cancel()) or the thread should_suspend
"""
self._is_not_asleep.clear()
self._is_not_asleep.wait(timeout)
self._is_not_asleep.set()
def is_asleep(self):
"""
Check from another thread whether the current thread is asleep
"""
return not self._is_not_asleep.is_set()
def unblock_callers(self):
"""
Ensures that any other thread that requested this thread's suspension
is released
"""
self._suspension_reached.set()
class ProcessingQueue(Queue.Queue, object):
"""
Queue of queues that processes a queue completely before moving on to the
next queue. There's one queue per Section(). You need to initialize each
section with add_section(section) first.
Put tuples (count, item) into this queue, with count being the respective
position of the item in the queue, starting with 0 (zero).
(None, None) is the sentinel for a single queue being exhausted, added by
add_sentinel()
"""
def _init(self, maxsize):
self.queue = deque()
self._sections = deque()
self._queues = deque()
self._current_section = None
self._current_queue = None
# Item-index for the currently active queue
self._counter = 0
def _qsize(self):
return self._current_queue._qsize() if self._current_queue else 0
def _total_qsize(self):
return sum(q._qsize() for q in self._queues) if self._queues else 0
def put(self, item, block=True, timeout=None):
"""
PKC customization of Queue.put. item needs to be the tuple
(count [int], {'section': [Section], 'xml': [etree xml]})
"""
self.not_full.acquire()
try:
if self.maxsize > 0:
if not block:
if self._total_qsize() == self.maxsize:
raise Queue.Full
elif timeout is None:
while self._total_qsize() == self.maxsize:
self.not_full.wait()
elif timeout < 0:
raise ValueError("'timeout' must be a non-negative number")
else:
endtime = _time() + timeout
while self._total_qsize() == self.maxsize:
remaining = endtime - _time()
if remaining <= 0.0:
raise Queue.Full
self.not_full.wait(remaining)
self._put(item)
self.unfinished_tasks += 1
self.not_empty.notify()
finally:
self.not_full.release()
def _put(self, item):
for i, section in enumerate(self._sections):
if item[1]['section'] == section:
self._queues[i]._put(item)
break
else:
raise RuntimeError('Could not find section for item %s' % item[1])
def add_sentinel(self, section):
"""
Adds a new empty section as a sentinel. Call with an empty Section()
object. Call this method immediately after having added all sections
with add_section().
Once the get()-method returns None, you've received the sentinel and
you've thus exhausted the queue
"""
self.not_full.acquire()
try:
section.number_of_items = 1
self._add_section(section)
# Add the actual sentinel to the queue we just added
self._queues[-1]._put((None, None))
self.unfinished_tasks += 1
self.not_empty.notify()
finally:
self.not_full.release()
def add_section(self, section):
"""
Add a new Section() to this Queue. Each section will be entirely
processed before moving on to the next section.
Be sure to set section.number_of_items correctly as it will signal
when processing is completely done for a specific section!
"""
self.mutex.acquire()
try:
self._add_section(section)
finally:
self.mutex.release()
def _add_section(self, section):
self._sections.append(section)
self._queues.append(
OrderedQueue() if section.plex_type == v.PLEX_TYPE_ALBUM
else Queue.Queue())
if self._current_section is None:
self._activate_next_section()
def _init_next_section(self):
"""
Call only when a section has been completely exhausted
"""
self._sections.popleft()
self._queues.popleft()
self._activate_next_section()
def _activate_next_section(self):
self._counter = 0
self._current_section = self._sections[0] if self._sections else None
self._current_queue = self._queues[0] if self._queues else None
def _get(self):
item = self._current_queue._get()
self._counter += 1
if self._counter == self._current_section.number_of_items:
self._init_next_section()
return item[1]
class OrderedQueue(Queue.PriorityQueue, object): class OrderedQueue(Queue.PriorityQueue, object):
@ -147,58 +246,24 @@ class OrderedQueue(Queue.PriorityQueue, object):
(index, item) (index, item)
where index=-1 is the item that will be returned first. The Queue will block where index=-1 is the item that will be returned first. The Queue will block
until index=-1, 0, 1, 2, 3, ... is then made available until index=-1, 0, 1, 2, 3, ... is then made available
maxsize will be rather fuzzy, as _qsize returns 0 if we're still waiting
for the next smalles index. put() thus might not block always when it
should.
""" """
def __init__(self, maxsize=0): def __init__(self, maxsize=0):
self.next_index = 0
super(OrderedQueue, self).__init__(maxsize) super(OrderedQueue, self).__init__(maxsize)
self.smallest = -1
self.not_next_item = threading.Condition(self.mutex)
def _put(self, item, heappush=heapq.heappush): def _qsize(self, len=len):
heappush(self.queue, item)
if item[0] == self.smallest:
self.not_next_item.notify()
def get(self, block=True, timeout=None):
"""Remove and return an item from the queue.
If optional args 'block' is true and 'timeout' is None (the default),
block if necessary until an item is available. If 'timeout' is
a non-negative number, it blocks at most 'timeout' seconds and raises
the Empty exception if no item was available within that time.
Otherwise ('block' is false), return an item if one is immediately
available, else raise the Empty exception ('timeout' is ignored
in that case).
"""
self.not_empty.acquire()
try: try:
if not block: return len(self.queue) if self.queue[0][0] == self.next_index else 0
if not self._qsize() or self.queue[0][0] != self.smallest: except IndexError:
raise Queue.Empty return 0
elif timeout is None:
while not self._qsize(): def _get(self, heappop=heapq.heappop):
self.not_empty.wait() self.next_index += 1
while self.queue[0][0] != self.smallest: return heappop(self.queue)
self.not_next_item.wait()
elif timeout < 0:
raise ValueError("'timeout' must be a non-negative number")
else:
endtime = Queue._time() + timeout
while not self._qsize():
remaining = endtime - Queue._time()
if remaining <= 0.0:
raise Queue.Empty
self.not_empty.wait(remaining)
while self.queue[0][0] != self.smallest:
remaining = endtime - Queue._time()
if remaining <= 0.0:
raise Queue.Empty
self.not_next_item.wait(remaining)
item = self._get()
self.smallest += 1
self.not_full.notify()
return item
finally:
self.not_empty.release()
class Tasks(list): class Tasks(list):
@ -239,13 +304,18 @@ class Task(object):
def cancel(self): def cancel(self):
self._canceled = True self._canceled = True
def isCanceled(self): def should_cancel(self):
return self._canceled or xbmc.abortRequested return self._canceled or xbmc.abortRequested
def isValid(self): def isValid(self):
return not self.finished and not self._canceled return not self.finished and not self._canceled
class ShutdownSentinel(Task):
def run(self):
pass
class FunctionAsTask(Task): class FunctionAsTask(Task):
def __init__(self, function, callback, *args, **kwargs): def __init__(self, function, callback, *args, **kwargs):
self._function = function self._function = function
@ -272,7 +342,7 @@ class MutablePriorityQueue(Queue.PriorityQueue):
lowest = self.queue and min(self.queue) or None lowest = self.queue and min(self.queue) or None
except Exception: except Exception:
lowest = None lowest = None
utils.ERROR() utils.ERROR(notify=True)
finally: finally:
self.mutex.release() self.mutex.release()
return lowest return lowest
@ -293,7 +363,7 @@ class BackgroundWorker(object):
try: try:
task._run() task._run()
except Exception: except Exception:
utils.ERROR() utils.ERROR(notify=True)
def abort(self): def abort(self):
self._abort = True self._abort = True
@ -323,13 +393,13 @@ class BackgroundWorker(object):
except Queue.Empty: except Queue.Empty:
LOG.debug('(%s): Idle', self.name) LOG.debug('(%s): Idle', self.name)
def shutdown(self): def shutdown(self, block=True):
self.abort() self.abort()
if self._task: if self._task:
self._task.cancel() self._task.cancel()
if self._thread and self._thread.isAlive(): if block and self._thread and self._thread.isAlive():
LOG.debug('thread (%s): Waiting...', self.name) LOG.debug('thread (%s): Waiting...', self.name)
self._thread.join() self._thread.join()
LOG.debug('thread (%s): Done', self.name) LOG.debug('thread (%s): Done', self.name)
@ -344,16 +414,17 @@ class NonstoppingBackgroundWorker(BackgroundWorker):
super(NonstoppingBackgroundWorker, self).__init__(queue, name) super(NonstoppingBackgroundWorker, self).__init__(queue, name)
def _queueLoop(self): def _queueLoop(self):
LOG.debug('Starting Worker %s', self.name)
while not self.aborted(): while not self.aborted():
try: self._task = self._queue.get()
self._task = self._queue.get_nowait() if self._task is ShutdownSentinel:
self._working = True break
self._runTask(self._task) self._working = True
self._working = False self._runTask(self._task)
self._queue.task_done() self._working = False
self._task = None self._queue.task_done()
except Queue.Empty: self._task = None
app.APP.monitor.waitForAbort(0.05) LOG.debug('Exiting Worker %s', self.name)
def working(self): def working(self):
return self._working return self._working
@ -365,7 +436,10 @@ class BackgroundThreader:
self._queue = MutablePriorityQueue() self._queue = MutablePriorityQueue()
self._abort = False self._abort = False
self.priority = -1 self.priority = -1
self.workers = [worker(self._queue, 'queue.{0}:worker.{1}'.format(self.name, x)) for x in range(worker_count)] self.workers = [
worker(self._queue, 'queue.{0}:worker.{1}'.format(self.name, x))
for x in range(worker_count)
]
def _nextPriority(self): def _nextPriority(self):
self.priority += 1 self.priority += 1
@ -380,11 +454,11 @@ class BackgroundThreader:
def aborted(self): def aborted(self):
return self._abort or xbmc.abortRequested return self._abort or xbmc.abortRequested
def shutdown(self): def shutdown(self, block=True):
self.abort() self.abort()
self.addTasksToFront([ShutdownSentinel() for _ in self.workers])
for w in self.workers: for w in self.workers:
w.shutdown() w.shutdown(block)
def addTask(self, task): def addTask(self, task):
task.priority = self._nextPriority() task.priority = self._nextPriority()
@ -437,7 +511,9 @@ class BackgroundThreader:
class ThreaderManager: class ThreaderManager:
def __init__(self, worker=BackgroundWorker, worker_count=6): def __init__(self,
worker=NonstoppingBackgroundWorker,
worker_count=WORKER_COUNT):
self.index = 0 self.index = 0
self.abandoned = [] self.abandoned = []
self._workerhandler = worker self._workerhandler = worker
@ -457,10 +533,10 @@ class ThreaderManager:
self.threader = BackgroundThreader(name=str(self.index), self.threader = BackgroundThreader(name=str(self.index),
worker=self._workerhandler) worker=self._workerhandler)
def shutdown(self): def shutdown(self, block=True):
self.threader.shutdown() self.threader.shutdown(block)
for a in self.abandoned: for a in self.abandoned:
a.shutdown() a.shutdown(block)
BGThreader = ThreaderManager() BGThreader = ThreaderManager()

25
resources/lib/db.py
View file

@ -6,6 +6,7 @@ from functools import wraps
from . import variables as v, app from . import variables as v, app
DB_WRITE_ATTEMPTS = 100 DB_WRITE_ATTEMPTS = 100
DB_CONNECTION_TIMEOUT = 10
class LockedDatabase(Exception): class LockedDatabase(Exception):
@ -52,39 +53,39 @@ def catch_operationalerrors(method):
return wrapper return wrapper
def _initial_db_connection_setup(conn, wal_mode): def _initial_db_connection_setup(conn):
""" """
Set-up DB e.g. for WAL journal mode, if that hasn't already been done Set-up DB e.g. for WAL journal mode, if that hasn't already been done
before. Also start a transaction before. Also start a transaction
""" """
if wal_mode: conn.execute('PRAGMA journal_mode = WAL;')
conn.execute('PRAGMA journal_mode=WAL;') conn.execute('PRAGMA cache_size = -8000;')
conn.execute('PRAGMA cache_size = -8000;') conn.execute('PRAGMA synchronous = NORMAL;')
conn.execute('PRAGMA synchronous=NORMAL;')
conn.execute('BEGIN') conn.execute('BEGIN')
def connect(media_type=None, wal_mode=True): def connect(media_type=None):
""" """
Open a connection to the Kodi database. Open a connection to the Kodi database.
media_type: 'video' (standard if not passed), 'plex', 'music', 'texture' media_type: 'video' (standard if not passed), 'plex', 'music', 'texture'
Pass wal_mode=False if you want the standard (and slower) sqlite
journal_mode, e.g. when wiping entire tables. Useful if you do NOT want
concurrent access to DB for both PKC and Kodi
""" """
if media_type == "plex": if media_type == "plex":
db_path = v.DB_PLEX_PATH db_path = v.DB_PLEX_PATH
elif media_type == 'plex-copy':
db_path = v.DB_PLEX_COPY_PATH
elif media_type == "music": elif media_type == "music":
db_path = v.DB_MUSIC_PATH db_path = v.DB_MUSIC_PATH
elif media_type == "texture": elif media_type == "texture":
db_path = v.DB_TEXTURE_PATH db_path = v.DB_TEXTURE_PATH
else: else:
db_path = v.DB_VIDEO_PATH db_path = v.DB_VIDEO_PATH
conn = sqlite3.connect(db_path, timeout=30.0) conn = sqlite3.connect(db_path,
timeout=DB_CONNECTION_TIMEOUT,
isolation_level=None)
attempts = DB_WRITE_ATTEMPTS attempts = DB_WRITE_ATTEMPTS
while True: while True:
try: try:
_initial_db_connection_setup(conn, wal_mode) _initial_db_connection_setup(conn)
except sqlite3.OperationalError as err: except sqlite3.OperationalError as err:
if 'database is locked' not in err: if 'database is locked' not in err:
# Not an error we want to catch, so reraise it # Not an error we want to catch, so reraise it
@ -95,7 +96,7 @@ def connect(media_type=None, wal_mode=True):
raise LockedDatabase('Database is locked') raise LockedDatabase('Database is locked')
if app.APP.monitor.waitForAbort(0.05): if app.APP.monitor.waitForAbort(0.05):
# PKC needs to quit # PKC needs to quit
return raise LockedDatabase('Database was locked and we need to exit')
else: else:
break break
return conn return conn

6
resources/lib/entrypoint.py
View file

@ -468,7 +468,7 @@ def watchlater():
def browse_plex(key=None, plex_type=None, section_id=None, synched=True, def browse_plex(key=None, plex_type=None, section_id=None, synched=True,
args=None, prompt=None): args=None, prompt=None, query=None):
""" """
Lists the content of a Plex folder, e.g. channels. Either pass in key (to Lists the content of a Plex folder, e.g. channels. Either pass in key (to
be used directly for PMS url {server}<key>) or the section_id be used directly for PMS url {server}<key>) or the section_id
@ -483,7 +483,9 @@ def browse_plex(key=None, plex_type=None, section_id=None, synched=True,
return return
app.init(entrypoint=True) app.init(entrypoint=True)
args = args or {} args = args or {}
if prompt: if query:
args['query'] = query
elif prompt:
prompt = utils.dialog('input', prompt) prompt = utils.dialog('input', prompt)
if prompt is None: if prompt is None:
# User cancelled # User cancelled

76
resources/lib/itemtypes/movies.py
View file

@ -5,7 +5,7 @@ from logging import getLogger
from .common import ItemBase from .common import ItemBase
from ..plex_api import API from ..plex_api import API
from .. import app, variables as v, plex_functions as PF from .. import app, variables as v, plex_functions as PF, utils
LOG = getLogger('PLEX.movies') LOG = getLogger('PLEX.movies')
@ -20,10 +20,10 @@ class Movie(ItemBase):
Process single movie Process single movie
""" """
api = API(xml) api = API(xml)
if not self.sync_this_item(api.library_section_id()): if not self.sync_this_item(section_id or api.library_section_id()):
LOG.debug('Skipping sync of %s %s: %s - section %s not synched to ' LOG.debug('Skipping sync of %s %s: %s - section %s not synched to '
'Kodi', api.plex_type, api.plex_id, api.title(), 'Kodi', api.plex_type, api.plex_id, api.title(),
api.library_section_id()) section_id or api.library_section_id())
return return
plex_id = api.plex_id plex_id = api.plex_id
movie = self.plexdb.movie(plex_id) movie = self.plexdb.movie(plex_id)
@ -54,7 +54,7 @@ class Movie(ItemBase):
else: else:
# Network share # Network share
filename = playurl.rsplit("/", 1)[1] filename = playurl.rsplit("/", 1)[1]
path = playurl.replace(filename, "") path = utils.rreplace(playurl, filename, "", 1)
kodi_pathid = self.kodidb.add_path(path, kodi_pathid = self.kodidb.add_path(path,
content='movies', content='movies',
scraper='metadata.local') scraper='metadata.local')
@ -74,31 +74,21 @@ class Movie(ItemBase):
api.date_created()) api.date_created())
if file_id != old_kodi_fileid: if file_id != old_kodi_fileid:
self.kodidb.remove_file(old_kodi_fileid) self.kodidb.remove_file(old_kodi_fileid)
rating_id = self.kodidb.get_ratingid(kodi_id, rating_id = self.kodidb.update_ratings(kodi_id,
v.KODI_TYPE_MOVIE) v.KODI_TYPE_MOVIE,
self.kodidb.update_ratings(kodi_id, "default",
v.KODI_TYPE_MOVIE, api.rating(),
"default", api.votecount())
api.rating(),
api.votecount(),
rating_id)
# update new uniqueid Kodi 17
if api.provider('imdb') is not None: if api.provider('imdb') is not None:
uniqueid = self.kodidb.get_uniqueid(kodi_id, uniqueid = self.kodidb.update_uniqueid(kodi_id,
v.KODI_TYPE_MOVIE) v.KODI_TYPE_MOVIE,
self.kodidb.update_uniqueid(kodi_id, 'imdb',
v.KODI_TYPE_MOVIE, api.provider('imdb'))
api.provider('imdb'),
"imdb",
uniqueid)
elif api.provider('tmdb') is not None: elif api.provider('tmdb') is not None:
uniqueid = self.kodidb.get_uniqueid(kodi_id, uniqueid = self.kodidb.update_uniqueid(kodi_id,
v.KODI_TYPE_MOVIE) v.KODI_TYPE_MOVIE,
self.kodidb.update_uniqueid(kodi_id, 'tmdb',
v.KODI_TYPE_MOVIE, api.provider('tmdb'))
api.provider('tmdb'),
"tmdb",
uniqueid)
else: else:
self.kodidb.remove_uniqueid(kodi_id, v.KODI_TYPE_MOVIE) self.kodidb.remove_uniqueid(kodi_id, v.KODI_TYPE_MOVIE)
uniqueid = -1 uniqueid = -1
@ -114,27 +104,21 @@ class Movie(ItemBase):
file_id = self.kodidb.add_file(filename, file_id = self.kodidb.add_file(filename,
kodi_pathid, kodi_pathid,
api.date_created()) api.date_created())
rating_id = self.kodidb.add_ratingid() rating_id = self.kodidb.add_ratings(kodi_id,
self.kodidb.add_ratings(rating_id, v.KODI_TYPE_MOVIE,
kodi_id, "default",
v.KODI_TYPE_MOVIE, api.rating(),
"default", api.votecount())
api.rating(),
api.votecount())
if api.provider('imdb') is not None: if api.provider('imdb') is not None:
uniqueid = self.kodidb.add_uniqueid_id() uniqueid = self.kodidb.add_uniqueid(kodi_id,
self.kodidb.add_uniqueid(uniqueid, v.KODI_TYPE_MOVIE,
kodi_id, api.provider('imdb'),
v.KODI_TYPE_MOVIE, "imdb")
api.provider('imdb'),
"imdb")
elif api.provider('tmdb') is not None: elif api.provider('tmdb') is not None:
uniqueid = self.kodidb.add_uniqueid_id() uniqueid = self.kodidb.add_uniqueid(kodi_id,
self.kodidb.add_uniqueid(uniqueid, v.KODI_TYPE_MOVIE,
kodi_id, api.provider('tmdb'),
v.KODI_TYPE_MOVIE, "tmdb")
api.provider('tmdb'),
"tmdb")
else: else:
uniqueid = -1 uniqueid = -1
self.kodidb.add_people(kodi_id, self.kodidb.add_people(kodi_id,

18
resources/lib/itemtypes/music.py
View file

@ -7,7 +7,7 @@ from .common import ItemBase
from ..plex_api import API from ..plex_api import API
from ..plex_db import PlexDB, PLEXDB_LOCK from ..plex_db import PlexDB, PLEXDB_LOCK
from ..kodi_db import KodiMusicDB, KODIDB_LOCK from ..kodi_db import KodiMusicDB, KODIDB_LOCK
from .. import plex_functions as PF, db, timing, app, variables as v from .. import plex_functions as PF, db, timing, app, variables as v, utils
LOG = getLogger('PLEX.music') LOG = getLogger('PLEX.music')
@ -159,10 +159,10 @@ class Artist(MusicMixin, ItemBase):
Process a single artist Process a single artist
""" """
api = API(xml) api = API(xml)
if not self.sync_this_item(api.library_section_id()): if not self.sync_this_item(section_id or api.library_section_id()):
LOG.debug('Skipping sync of %s %s: %s - section %s not synched to ' LOG.debug('Skipping sync of %s %s: %s - section %s not synched to '
'Kodi', api.plex_type, api.plex_id, api.title(), 'Kodi', api.plex_type, api.plex_id, api.title(),
api.library_section_id()) section_id or api.library_section_id())
return return
plex_id = api.plex_id plex_id = api.plex_id
artist = self.plexdb.artist(plex_id) artist = self.plexdb.artist(plex_id)
@ -225,6 +225,11 @@ class Album(MusicMixin, ItemBase):
avoid infinite loops avoid infinite loops
""" """
api = API(xml) api = API(xml)
if not self.sync_this_item(section_id or api.library_section_id()):
LOG.debug('Skipping sync of %s %s: %s - section %s not synched to '
'Kodi', api.plex_type, api.plex_id, api.title(),
section_id or api.library_section_id())
return
plex_id = api.plex_id plex_id = api.plex_id
album = self.plexdb.album(plex_id) album = self.plexdb.album(plex_id)
if album: if album:
@ -387,6 +392,11 @@ class Song(MusicMixin, ItemBase):
Process single song/track Process single song/track
""" """
api = API(xml) api = API(xml)
if not self.sync_this_item(section_id or api.library_section_id()):
LOG.debug('Skipping sync of %s %s: %s - section %s not synched to '
'Kodi', api.plex_type, api.plex_id, api.title(),
section_id or api.library_section_id())
return
plex_id = api.plex_id plex_id = api.plex_id
song = self.plexdb.song(plex_id) song = self.plexdb.song(plex_id)
if song: if song:
@ -529,7 +539,7 @@ class Song(MusicMixin, ItemBase):
else: else:
# Network share # Network share
filename = playurl.rsplit("/", 1)[1] filename = playurl.rsplit("/", 1)[1]
path = playurl.replace(filename, "") path = utils.rreplace(playurl, filename, "", 1)
if do_indirect: if do_indirect:
# Plex works a bit differently # Plex works a bit differently
path = "%s%s" % (app.CONN.server, xml[0][0].get('key')) path = "%s%s" % (app.CONN.server, xml[0][0].get('key'))

151
resources/lib/itemtypes/tvshows.py
View file

@ -5,7 +5,7 @@ from logging import getLogger
from .common import ItemBase, process_path from .common import ItemBase, process_path
from ..plex_api import API from ..plex_api import API
from .. import plex_functions as PF, app, variables as v from .. import plex_functions as PF, app, variables as v, utils
LOG = getLogger('PLEX.tvshows') LOG = getLogger('PLEX.tvshows')
@ -148,10 +148,10 @@ class Show(TvShowMixin, ItemBase):
Process a single show Process a single show
""" """
api = API(xml) api = API(xml)
if not self.sync_this_item(api.library_section_id()): if not self.sync_this_item(section_id or api.library_section_id()):
LOG.debug('Skipping sync of %s %s: %s - section %s not synched to ' LOG.debug('Skipping sync of %s %s: %s - section %s not synched to '
'Kodi', api.plex_type, api.plex_id, api.title(), 'Kodi', api.plex_type, api.plex_id, api.title(),
api.library_section_id()) section_id or api.library_section_id())
return return
plex_id = api.plex_id plex_id = api.plex_id
show = self.plexdb.show(plex_id) show = self.plexdb.show(plex_id)
@ -189,29 +189,21 @@ class Show(TvShowMixin, ItemBase):
if update_item: if update_item:
LOG.info("UPDATE tvshow plex_id: %s - %s", plex_id, api.title()) LOG.info("UPDATE tvshow plex_id: %s - %s", plex_id, api.title())
# update new ratings Kodi 17 # update new ratings Kodi 17
rating_id = self.kodidb.get_ratingid(kodi_id, v.KODI_TYPE_SHOW) rating_id = self.kodidb.update_ratings(kodi_id,
self.kodidb.update_ratings(kodi_id, v.KODI_TYPE_SHOW,
v.KODI_TYPE_SHOW, "default",
"default", api.rating(),
api.rating(), api.votecount())
api.votecount(),
rating_id)
if api.provider('tvdb') is not None: if api.provider('tvdb') is not None:
uniqueid = self.kodidb.get_uniqueid(kodi_id, uniqueid = self.kodidb.update_uniqueid(kodi_id,
v.KODI_TYPE_SHOW) v.KODI_TYPE_SHOW,
self.kodidb.update_uniqueid(kodi_id, 'tvdb',
v.KODI_TYPE_SHOW, api.provider('tvdb'))
api.provider('tvdb'),
'tvdb',
uniqueid)
elif api.provider('tmdb') is not None: elif api.provider('tmdb') is not None:
uniqueid = self.kodidb.get_uniqueid(kodi_id, uniqueid = self.kodidb.update_uniqueid(kodi_id,
v.KODI_TYPE_SHOW) v.KODI_TYPE_SHOW,
self.kodidb.update_uniqueid(kodi_id, 'tmdb',
v.KODI_TYPE_SHOW, api.provider('tmdb'))
api.provider('tmdb'),
'tmdb',
uniqueid)
else: else:
self.kodidb.remove_uniqueid(kodi_id, v.KODI_TYPE_SHOW) self.kodidb.remove_uniqueid(kodi_id, v.KODI_TYPE_SHOW)
uniqueid = -1 uniqueid = -1
@ -239,27 +231,21 @@ class Show(TvShowMixin, ItemBase):
LOG.info("ADD tvshow plex_id: %s - %s", plex_id, api.title()) LOG.info("ADD tvshow plex_id: %s - %s", plex_id, api.title())
# Link the path # Link the path
self.kodidb.add_showlinkpath(kodi_id, kodi_pathid) self.kodidb.add_showlinkpath(kodi_id, kodi_pathid)
rating_id = self.kodidb.get_ratingid(kodi_id, v.KODI_TYPE_SHOW) rating_id = self.kodidb.add_ratings(kodi_id,
self.kodidb.add_ratings(rating_id, v.KODI_TYPE_SHOW,
kodi_id, "default",
v.KODI_TYPE_SHOW, api.rating(),
"default", api.votecount())
api.rating(),
api.votecount())
if api.provider('tvdb'): if api.provider('tvdb'):
uniqueid = self.kodidb.add_uniqueid_id() uniqueid = self.kodidb.add_uniqueid(kodi_id,
self.kodidb.add_uniqueid(uniqueid, v.KODI_TYPE_SHOW,
kodi_id, api.provider('tvdb'),
v.KODI_TYPE_SHOW, 'tvdb')
api.provider('tvdb'),
'tvdb')
if api.provider('tmdb'): if api.provider('tmdb'):
uniqueid = self.kodidb.add_uniqueid_id() uniqueid = self.kodidb.add_uniqueid(kodi_id,
self.kodidb.add_uniqueid(uniqueid, v.KODI_TYPE_SHOW,
kodi_id, api.provider('tmdb'),
v.KODI_TYPE_SHOW, 'tmdb')
api.provider('tmdb'),
'tmdb')
else: else:
uniqueid = -1 uniqueid = -1
self.kodidb.add_people(kodi_id, self.kodidb.add_people(kodi_id,
@ -303,10 +289,10 @@ class Season(TvShowMixin, ItemBase):
Process a single season of a certain tv show Process a single season of a certain tv show
""" """
api = API(xml) api = API(xml)
if not self.sync_this_item(api.library_section_id()): if not self.sync_this_item(section_id or api.library_section_id()):
LOG.debug('Skipping sync of %s %s: %s - section %s not synched to ' LOG.debug('Skipping sync of %s %s: %s - section %s not synched to '
'Kodi', api.plex_type, api.plex_id, api.title(), 'Kodi', api.plex_type, api.plex_id, api.title(),
api.library_section_id()) section_id or api.library_section_id())
return return
plex_id = api.plex_id plex_id = api.plex_id
season = self.plexdb.season(plex_id) season = self.plexdb.season(plex_id)
@ -372,10 +358,10 @@ class Episode(TvShowMixin, ItemBase):
Process single episode Process single episode
""" """
api = API(xml) api = API(xml)
if not self.sync_this_item(api.library_section_id()): if not self.sync_this_item(section_id or api.library_section_id()):
LOG.debug('Skipping sync of %s %s: %s - section %s not synched to ' LOG.debug('Skipping sync of %s %s: %s - section %s not synched to '
'Kodi', api.plex_type, api.plex_id, api.title(), 'Kodi', api.plex_type, api.plex_id, api.title(),
api.library_section_id()) section_id or api.library_section_id())
return return
plex_id = api.plex_id plex_id = api.plex_id
episode = self.plexdb.episode(plex_id) episode = self.plexdb.episode(plex_id)
@ -448,7 +434,7 @@ class Episode(TvShowMixin, ItemBase):
else: else:
# Network share # Network share
filename = playurl.rsplit("/", 1)[1] filename = playurl.rsplit("/", 1)[1]
path = playurl.replace(filename, "") path = utils.rreplace(playurl, filename, "", 1)
parent_path_id = self.kodidb.parent_path_id(path) parent_path_id = self.kodidb.parent_path_id(path)
kodi_pathid = self.kodidb.add_path(path, kodi_pathid = self.kodidb.add_path(path,
id_parent_path=parent_path_id) id_parent_path=parent_path_id)
@ -489,30 +475,21 @@ class Episode(TvShowMixin, ItemBase):
self.kodidb.remove_file(old_kodi_fileid) self.kodidb.remove_file(old_kodi_fileid)
if not app.SYNC.direct_paths: if not app.SYNC.direct_paths:
self.kodidb.remove_file(old_kodi_fileid_2) self.kodidb.remove_file(old_kodi_fileid_2)
ratingid = self.kodidb.get_ratingid(kodi_id, ratingid = self.kodidb.update_ratings(kodi_id,
v.KODI_TYPE_EPISODE) v.KODI_TYPE_EPISODE,
self.kodidb.update_ratings(kodi_id, "default",
v.KODI_TYPE_EPISODE, api.rating(),
"default", api.votecount())
api.rating(),
api.votecount(),
ratingid)
if api.provider('tvdb'): if api.provider('tvdb'):
uniqueid = self.kodidb.get_uniqueid(kodi_id, uniqueid = self.kodidb.update_uniqueid(kodi_id,
v.KODI_TYPE_EPISODE) v.KODI_TYPE_EPISODE,
self.kodidb.update_uniqueid(kodi_id, 'tvdb',
v.KODI_TYPE_EPISODE, api.provider('tvdb'))
api.provider('tvdb'),
"tvdb",
uniqueid)
elif api.provider('tmdb'): elif api.provider('tmdb'):
uniqueid = self.kodidb.get_uniqueid(kodi_id, uniqueid = self.kodidb.update_uniqueid(kodi_id,
v.KODI_TYPE_EPISODE) v.KODI_TYPE_EPISODE,
self.kodidb.update_uniqueid(kodi_id, 'tmdb',
v.KODI_TYPE_EPISODE, api.provider('tmdb'))
api.provider('tmdb'),
"tmdb",
uniqueid)
else: else:
self.kodidb.remove_uniqueid(kodi_id, v.KODI_TYPE_EPISODE) self.kodidb.remove_uniqueid(kodi_id, v.KODI_TYPE_EPISODE)
uniqueid = -1 uniqueid = -1
@ -537,6 +514,7 @@ class Episode(TvShowMixin, ItemBase):
airs_before_episode, airs_before_episode,
playurl, playurl,
kodi_pathid, kodi_pathid,
uniqueid,
kodi_fileid, # and NOT kodi_fileid_2 kodi_fileid, # and NOT kodi_fileid_2
parent_id, parent_id,
api.userrating(), api.userrating(),
@ -577,27 +555,21 @@ class Episode(TvShowMixin, ItemBase):
else: else:
kodi_fileid_2 = None kodi_fileid_2 = None
rating_id = self.kodidb.add_ratingid() rating_id = self.kodidb.add_ratings(kodi_id,
self.kodidb.add_ratings(rating_id, v.KODI_TYPE_EPISODE,
kodi_id, "default",
v.KODI_TYPE_EPISODE, api.rating(),
"default", api.votecount())
api.rating(),
api.votecount())
if api.provider('tvdb'): if api.provider('tvdb'):
uniqueid = self.kodidb.add_uniqueid_id() uniqueid = self.kodidb.add_uniqueid(kodi_id,
self.kodidb.add_uniqueid(uniqueid, v.KODI_TYPE_EPISODE,
kodi_id, api.provider('tvdb'),
v.KODI_TYPE_EPISODE, "tvdb")
api.provider('tvdb'),
"tvdb")
elif api.provider('tmdb'): elif api.provider('tmdb'):
uniqueid = self.kodidb.add_uniqueid_id() uniqueid = self.kodidb.add_uniqueid(kodi_id,
self.kodidb.add_uniqueid(uniqueid, v.KODI_TYPE_EPISODE,
kodi_id, api.provider('tmdb'),
v.KODI_TYPE_EPISODE, "tmdb")
api.provider('tmdb'),
"tmdb")
else: else:
uniqueid = -1 uniqueid = -1
self.kodidb.add_people(kodi_id, self.kodidb.add_people(kodi_id,
@ -624,6 +596,7 @@ class Episode(TvShowMixin, ItemBase):
airs_before_episode, airs_before_episode,
playurl, playurl,
kodi_pathid, kodi_pathid,
uniqueid,
parent_id, parent_id,
api.userrating()) api.userrating())
self.kodidb.set_resume(kodi_fileid, self.kodidb.set_resume(kodi_fileid,

6
resources/lib/kodi_db/__init__.py
View file

@ -62,7 +62,7 @@ def setup_kodi_default_entries():
def reset_cached_images(): def reset_cached_images():
LOG.info('Resetting cached artwork') LOG.info('Resetting cached artwork')
LOG.debug('Resetting the Kodi texture DB') LOG.debug('Resetting the Kodi texture DB')
with KodiTextureDB(wal_mode=False) as kodidb: with KodiTextureDB() as kodidb:
kodidb.wipe() kodidb.wipe()
LOG.debug('Deleting all cached image files') LOG.debug('Deleting all cached image files')
path = path_ops.translate_path('special://thumbnails/') path = path_ops.translate_path('special://thumbnails/')
@ -91,11 +91,11 @@ def wipe_dbs(music=True):
""" """
LOG.warn('Wiping Kodi databases!') LOG.warn('Wiping Kodi databases!')
LOG.info('Wiping Kodi video database') LOG.info('Wiping Kodi video database')
with KodiVideoDB(wal_mode=False) as kodidb: with KodiVideoDB() as kodidb:
kodidb.wipe() kodidb.wipe()
if music: if music:
LOG.info('Wiping Kodi music database') LOG.info('Wiping Kodi music database')
with KodiMusicDB(wal_mode=False) as kodidb: with KodiMusicDB() as kodidb:
kodidb.wipe() kodidb.wipe()
reset_cached_images() reset_cached_images()
setup_kodi_default_entries() setup_kodi_default_entries()

9
resources/lib/kodi_db/common.py
View file

@ -15,11 +15,9 @@ class KodiDBBase(object):
Kodi database methods used for all types of items Kodi database methods used for all types of items
""" """
def __init__(self, texture_db=False, kodiconn=None, artconn=None, def __init__(self, texture_db=False, kodiconn=None, artconn=None,
lock=True, wal_mode=True): lock=True):
""" """
Allows direct use with a cursor instead of context mgr Allows direct use with a cursor instead of context mgr
Pass wal_mode=False if you want the standard sqlite journal_mode, e.g.
when wiping entire tables
""" """
self._texture_db = texture_db self._texture_db = texture_db
self.lock = lock self.lock = lock
@ -27,14 +25,13 @@ class KodiDBBase(object):
self.cursor = self.kodiconn.cursor() if self.kodiconn else None self.cursor = self.kodiconn.cursor() if self.kodiconn else None
self.artconn = artconn self.artconn = artconn
self.artcursor = self.artconn.cursor() if self.artconn else None self.artcursor = self.artconn.cursor() if self.artconn else None
self.wal_mode = wal_mode
def __enter__(self): def __enter__(self):
if self.lock: if self.lock:
KODIDB_LOCK.acquire() KODIDB_LOCK.acquire()
self.kodiconn = db.connect(self.db_kind, self.wal_mode) self.kodiconn = db.connect(self.db_kind)
self.cursor = self.kodiconn.cursor() self.cursor = self.kodiconn.cursor()
self.artconn = db.connect('texture', self.wal_mode) if self._texture_db \ self.artconn = db.connect('texture') if self._texture_db \
else None else None
self.artcursor = self.artconn.cursor() if self._texture_db else None self.artcursor = self.artconn.cursor() if self._texture_db else None
return self return self

36
resources/lib/kodi_db/music.py
View file

@ -25,13 +25,9 @@ class KodiMusicDB(common.KodiDBBase):
try: try:
pathid = self.cursor.fetchone()[0] pathid = self.cursor.fetchone()[0]
except TypeError: except TypeError:
self.cursor.execute("SELECT COALESCE(MAX(idPath),0) FROM path") self.cursor.execute('INSERT INTO path(strPath, strHash) VALUES (?, ?)',
pathid = self.cursor.fetchone()[0] + 1 (path, '123'))
self.cursor.execute(''' pathid = self.cursor.lastrowid
INSERT INTO path(idPath, strPath, strHash)
VALUES (?, ?, ?)
''',
(pathid, path, '123'))
return pathid return pathid
@db.catch_operationalerrors @db.catch_operationalerrors
@ -382,10 +378,9 @@ class KodiMusicDB(common.KodiDBBase):
genreid = self.cursor.fetchone()[0] genreid = self.cursor.fetchone()[0]
except TypeError: except TypeError:
# Create the genre # Create the genre
self.cursor.execute('SELECT COALESCE(MAX(idGenre),0) FROM genre') self.cursor.execute('INSERT INTO genre(strGenre) VALUES(?)',
genreid = self.cursor.fetchone()[0] + 1 (genre, ))
self.cursor.execute('INSERT INTO genre(idGenre, strGenre) VALUES(?, ?)', genreid = self.cursor.lastrowid
(genreid, genre))
self.cursor.execute(''' self.cursor.execute('''
INSERT OR REPLACE INTO album_genre( INSERT OR REPLACE INTO album_genre(
idGenre, idGenre,
@ -403,10 +398,9 @@ class KodiMusicDB(common.KodiDBBase):
genreid = self.cursor.fetchone()[0] genreid = self.cursor.fetchone()[0]
except TypeError: except TypeError:
# Create the genre # Create the genre
self.cursor.execute('SELECT COALESCE(MAX(idGenre),0) FROM genre') self.cursor.execute('INSERT INTO genre(strGenre) VALUES (?)',
genreid = self.cursor.fetchone()[0] + 1 (genre, ))
self.cursor.execute('INSERT INTO genre(idGenre, strGenre) values(?, ?)', genreid = self.cursor.lastrowid
(genreid, genre))
self.cursor.execute(''' self.cursor.execute('''
INSERT OR REPLACE INTO song_genre( INSERT OR REPLACE INTO song_genre(
idGenre, idGenre,
@ -550,15 +544,11 @@ class KodiMusicDB(common.KodiDBBase):
except TypeError: except TypeError:
# Krypton has a dummy first entry idArtist: 1 strArtist: # Krypton has a dummy first entry idArtist: 1 strArtist:
# [Missing Tag] strMusicBrainzArtistID: Artist Tag Missing # [Missing Tag] strMusicBrainzArtistID: Artist Tag Missing
self.cursor.execute('SELECT COALESCE(MAX(idArtist),1) FROM artist')
artistid = self.cursor.fetchone()[0] + 1
self.cursor.execute(''' self.cursor.execute('''
INSERT INTO artist( INSERT INTO artist(strArtist, strMusicBrainzArtistID)
idArtist, VALUES (?, ?)
strArtist, ''', (name, musicbrainz))
strMusicBrainzArtistID) artistid = self.cursor.lastrowid
VALUES (?, ?, ?)
''', (artistid, name, musicbrainz))
else: else:
if artistname != name: if artistname != name:
self.cursor.execute('UPDATE artist SET strArtist = ? WHERE idArtist = ?', self.cursor.execute('UPDATE artist SET strArtist = ? WHERE idArtist = ?',

197
resources/lib/kodi_db/video.py
View file

@ -38,45 +38,22 @@ class KodiVideoDB(common.KodiDBBase):
For some reason, Kodi ignores this if done via itemtypes while e.g. For some reason, Kodi ignores this if done via itemtypes while e.g.
adding or updating items. (addPath method does NOT work) adding or updating items. (addPath method does NOT work)
""" """
path_id = self.get_path(MOVIE_PATH) for path, kind in ((MOVIE_PATH, 'movies'), (SHOW_PATH, 'tvshows')):
if path_id is None: path_id = self.get_path(path)
self.cursor.execute("SELECT COALESCE(MAX(idPath),0) FROM path") if path_id is None:
path_id = self.cursor.fetchone()[0] + 1 query = '''
query = ''' INSERT INTO path(strPath,
INSERT INTO path(idPath, strContent,
strPath, strScraper,
strContent, noUpdate,
strScraper, exclude)
noUpdate, VALUES (?, ?, ?, ?, ?)
exclude) '''
VALUES (?, ?, ?, ?, ?, ?) self.cursor.execute(query, (path,
''' kind,
self.cursor.execute(query, (path_id, 'metadata.local',
MOVIE_PATH, 1,
'movies', 0))
'metadata.local',
1,
0))
# And TV shows
path_id = self.get_path(SHOW_PATH)
if path_id is None:
self.cursor.execute("SELECT COALESCE(MAX(idPath),0) FROM path")
path_id = self.cursor.fetchone()[0] + 1
query = '''
INSERT INTO path(idPath,
strPath,
strContent,
strScraper,
noUpdate,
exclude)
VALUES (?, ?, ?, ?, ?, ?)
'''
self.cursor.execute(query, (path_id,
SHOW_PATH,
'tvshows',
'metadata.local',
1,
0))
@db.catch_operationalerrors @db.catch_operationalerrors
def parent_path_id(self, path): def parent_path_id(self, path):
@ -89,13 +66,12 @@ class KodiVideoDB(common.KodiDBBase):
path_ops.decode_path(path_ops.path.pardir))) path_ops.decode_path(path_ops.path.pardir)))
pathid = self.get_path(parentpath) pathid = self.get_path(parentpath)
if pathid is None: if pathid is None:
self.cursor.execute("SELECT COALESCE(MAX(idPath),0) FROM path")
pathid = self.cursor.fetchone()[0] + 1
self.cursor.execute(''' self.cursor.execute('''
INSERT INTO path(idPath, strPath, dateAdded) INSERT INTO path(strPath, dateAdded)
VALUES (?, ?, ?) VALUES (?, ?)
''', ''',
(pathid, parentpath, timing.kodi_now())) (parentpath, timing.kodi_now()))
pathid = self.cursor.lastrowid
if parentpath != path: if parentpath != path:
# In case we end up having media in the filesystem root, C:\ # In case we end up having media in the filesystem root, C:\
parent_id = self.parent_path_id(parentpath) parent_id = self.parent_path_id(parentpath)
@ -127,21 +103,19 @@ class KodiVideoDB(common.KodiDBBase):
try: try:
pathid = self.cursor.fetchone()[0] pathid = self.cursor.fetchone()[0]
except TypeError: except TypeError:
self.cursor.execute("SELECT COALESCE(MAX(idPath),0) FROM path")
pathid = self.cursor.fetchone()[0] + 1
self.cursor.execute(''' self.cursor.execute('''
INSERT INTO path( INSERT INTO path(
idPath,
strPath, strPath,
dateAdded, dateAdded,
idParentPath, idParentPath,
strContent, strContent,
strScraper, strScraper,
noUpdate) noUpdate)
VALUES (?, ?, ?, ?, ?, ?, ?) VALUES (?, ?, ?, ?, ?, ?)
''', ''',
(pathid, path, date_added, id_parent_path, (path, date_added, id_parent_path, content,
content, scraper, 1)) scraper, 1))
pathid = self.cursor.lastrowid
return pathid return pathid
def get_path(self, path): def get_path(self, path):
@ -161,18 +135,12 @@ class KodiVideoDB(common.KodiDBBase):
Adds the filename [unicode] to the table files if not already added Adds the filename [unicode] to the table files if not already added
and returns the idFile. and returns the idFile.
""" """
self.cursor.execute('SELECT COALESCE(MAX(idFile), 0) FROM files')
file_id = self.cursor.fetchone()[0] + 1
self.cursor.execute(''' self.cursor.execute('''
INSERT INTO files( INSERT INTO files(idPath, strFilename, dateAdded)
idFile, VALUES (?, ?, ?)
idPath,
strFilename,
dateAdded)
VALUES (?, ?, ?, ?)
''', ''',
(file_id, path_id, filename, date_added)) (path_id, filename, date_added))
return file_id return self.cursor.lastrowid
def modify_file(self, filename, path_id, date_added): def modify_file(self, filename, path_id, date_added):
self.cursor.execute('SELECT idFile FROM files WHERE idPath = ? AND strFilename = ?', self.cursor.execute('SELECT idFile FROM files WHERE idPath = ? AND strFilename = ?',
@ -261,11 +229,9 @@ class KodiVideoDB(common.KodiDBBase):
try: try:
entry_id = self.cursor.fetchone()[0] entry_id = self.cursor.fetchone()[0]
except TypeError: except TypeError:
self.cursor.execute('SELECT COALESCE(MAX(%s), %s) FROM %s' self.cursor.execute('INSERT INTO %s(name) VALUES(?)' % table,
% (key, first_id - 1, table)) (entry, ))
entry_id = self.cursor.fetchone()[0] + 1 entry_id = self.cursor.lastrowid
self.cursor.execute('INSERT INTO %s(%s, name) values(?, ?)'
% (table, key), (entry_id, entry))
finally: finally:
entry_ids.append(entry_id) entry_ids.append(entry_id)
# Now process the ids obtained from the names # Now process the ids obtained from the names
@ -458,10 +424,8 @@ class KodiVideoDB(common.KodiDBBase):
@db.catch_operationalerrors @db.catch_operationalerrors
def _new_actor_id(self, name, art_url): def _new_actor_id(self, name, art_url):
# Not yet in actor DB, add person # Not yet in actor DB, add person
self.cursor.execute('SELECT COALESCE(MAX(actor_id), 0) FROM actor') self.cursor.execute('INSERT INTO actor(name) VALUES (?)', (name, ))
actor_id = self.cursor.fetchone()[0] + 1 actor_id = self.cursor.lastrowid
self.cursor.execute('INSERT INTO actor(actor_id, name) VALUES (?, ?)',
(actor_id, name))
if art_url: if art_url:
self.add_art(art_url, actor_id, 'actor', 'thumb') self.add_art(art_url, actor_id, 'actor', 'thumb')
return actor_id return actor_id
@ -649,12 +613,8 @@ class KodiVideoDB(common.KodiDBBase):
(playcount or None, dateplayed, file_id)) (playcount or None, dateplayed, file_id))
# Set the resume bookmark # Set the resume bookmark
if resume_seconds: if resume_seconds:
self.cursor.execute(
'SELECT COALESCE(MAX(idBookmark), 0) FROM bookmark')
bookmark_id = self.cursor.fetchone()[0] + 1
self.cursor.execute(''' self.cursor.execute('''
INSERT INTO bookmark( INSERT INTO bookmark(
idBookmark,
idFile, idFile,
timeInSeconds, timeInSeconds,
totalTimeInSeconds, totalTimeInSeconds,
@ -662,9 +622,8 @@ class KodiVideoDB(common.KodiDBBase):
player, player,
playerState, playerState,
type) type)
VALUES (?, ?, ?, ?, ?, ?, ?, ?) VALUES (?, ?, ?, ?, ?, ?, ?)
''', (bookmark_id, ''', (file_id,
file_id,
resume_seconds, resume_seconds,
total_seconds, total_seconds,
'', '',
@ -682,10 +641,8 @@ class KodiVideoDB(common.KodiDBBase):
try: try:
tag_id = self.cursor.fetchone()[0] tag_id = self.cursor.fetchone()[0]
except TypeError: except TypeError:
self.cursor.execute("SELECT COALESCE(MAX(tag_id), 0) FROM tag") self.cursor.execute('INSERT INTO tag(name) VALUES(?)', (name, ))
tag_id = self.cursor.fetchone()[0] + 1 tag_id = self.cursor.lastrowid
self.cursor.execute('INSERT INTO tag(tag_id, name) VALUES(?, ?)',
(tag_id, name))
return tag_id return tag_id
@db.catch_operationalerrors @db.catch_operationalerrors
@ -717,10 +674,8 @@ class KodiVideoDB(common.KodiDBBase):
try: try:
setid = self.cursor.fetchone()[0] setid = self.cursor.fetchone()[0]
except TypeError: except TypeError:
self.cursor.execute("SELECT COALESCE(MAX(idSet), 0) FROM sets") self.cursor.execute('INSERT INTO sets(strSet) VALUES(?)', (set_name, ))
setid = self.cursor.fetchone()[0] + 1 setid = self.cursor.lastrowid
self.cursor.execute('INSERT INTO sets(idSet, strSet) VALUES(?, ?)',
(setid, set_name))
return setid return setid
@db.catch_operationalerrors @db.catch_operationalerrors
@ -768,19 +723,14 @@ class KodiVideoDB(common.KodiDBBase):
Adds a TV show season to the Kodi video DB or simply returns the ID, Adds a TV show season to the Kodi video DB or simply returns the ID,
if there already is an entry in the DB if there already is an entry in the DB
""" """
self.cursor.execute("SELECT COALESCE(MAX(idSeason),0) FROM seasons") self.cursor.execute('INSERT INTO seasons(idShow, season) VALUES (?, ?)',
seasonid = self.cursor.fetchone()[0] + 1 (showid, seasonnumber))
self.cursor.execute(''' return self.cursor.lastrowid
INSERT INTO seasons(idSeason, idShow, season)
VALUES (?, ?, ?)
''', (seasonid, showid, seasonnumber))
return seasonid
@db.catch_operationalerrors @db.catch_operationalerrors
def add_uniqueid(self, *args): def add_uniqueid(self, *args):
""" """
Feed with: Feed with:
uniqueid_id: int
media_id: int media_id: int
media_type: string media_type: string
value: string value: string
@ -788,39 +738,24 @@ class KodiVideoDB(common.KodiDBBase):
""" """
self.cursor.execute(''' self.cursor.execute('''
INSERT INTO uniqueid( INSERT INTO uniqueid(
uniqueid_id,
media_id, media_id,
media_type, media_type,
value, value,
type) type)
VALUES (?, ?, ?, ?, ?) VALUES (?, ?, ?, ?)
''', (args)) ''', (args))
return self.cursor.lastrowid
def add_uniqueid_id(self):
self.cursor.execute('SELECT COALESCE(MAX(uniqueid_id), 0) FROM uniqueid')
return self.cursor.fetchone()[0] + 1
def get_uniqueid(self, kodi_id, kodi_type):
"""
Returns the uniqueid_id
"""
self.cursor.execute('SELECT uniqueid_id FROM uniqueid WHERE media_id = ? AND media_type =?',
(kodi_id, kodi_type))
try:
return self.cursor.fetchone()[0]
except TypeError:
return self.add_uniqueid_id()
@db.catch_operationalerrors @db.catch_operationalerrors
def update_uniqueid(self, *args): def update_uniqueid(self, *args):
""" """
Pass in media_id, media_type, value, type, uniqueid_id Pass in value, media_id, media_type, type
""" """
self.cursor.execute(''' self.cursor.execute('''
UPDATE uniqueid INSERT OR REPLACE INTO uniqueid(media_id, media_type, type, value)
SET media_id = ?, media_type = ?, value = ?, type = ? VALUES(?, ?, ?, ?)
WHERE uniqueid_id = ?
''', (args)) ''', (args))
return self.cursor.lastrowid
@db.catch_operationalerrors @db.catch_operationalerrors
def remove_uniqueid(self, kodi_id, kodi_type): def remove_uniqueid(self, kodi_id, kodi_type):
@ -830,54 +765,36 @@ class KodiVideoDB(common.KodiDBBase):
self.cursor.execute('DELETE FROM uniqueid WHERE media_id = ? AND media_type = ?', self.cursor.execute('DELETE FROM uniqueid WHERE media_id = ? AND media_type = ?',
(kodi_id, kodi_type)) (kodi_id, kodi_type))
def add_ratingid(self):
self.cursor.execute('SELECT COALESCE(MAX(rating_id),0) FROM rating')
return self.cursor.fetchone()[0] + 1
def get_ratingid(self, kodi_id, kodi_type):
"""
Create if needed and return the unique rating_id from rating table
"""
self.cursor.execute('SELECT rating_id FROM rating WHERE media_id = ? AND media_type = ?',
(kodi_id, kodi_type))
try:
return self.cursor.fetchone()[0]
except TypeError:
return self.add_ratingid()
@db.catch_operationalerrors @db.catch_operationalerrors
def update_ratings(self, *args): def update_ratings(self, *args):
""" """
Feed with media_id, media_type, rating_type, rating, votes, rating_id Feed with media_id, media_type, rating_type, rating, votes, rating_id
""" """
self.cursor.execute(''' self.cursor.execute('''
UPDATE rating INSERT OR REPLACE INTO
SET media_id = ?, rating(media_id, media_type, rating_type, rating, votes)
media_type = ?, VALUES (?, ?, ?, ?, ?)
rating_type = ?,
rating = ?,
votes = ?
WHERE rating_id = ?
''', (args)) ''', (args))
return self.cursor.lastrowid
@db.catch_operationalerrors @db.catch_operationalerrors
def add_ratings(self, *args): def add_ratings(self, *args):
""" """
feed with: feed with:
rating_id, media_id, media_type, rating_type, rating, votes media_id, media_type, rating_type, rating, votes
rating_type = 'default' rating_type = 'default'
""" """
self.cursor.execute(''' self.cursor.execute('''
INSERT INTO rating( INSERT INTO rating(
rating_id,
media_id, media_id,
media_type, media_type,
rating_type, rating_type,
rating, rating,
votes) votes)
VALUES (?, ?, ?, ?, ?, ?) VALUES (?, ?, ?, ?, ?)
''', (args)) ''', (args))
return self.cursor.lastrowid
@db.catch_operationalerrors @db.catch_operationalerrors
def remove_ratings(self, kodi_id, kodi_type): def remove_ratings(self, kodi_id, kodi_type):
@ -917,10 +834,11 @@ class KodiVideoDB(common.KodiDBBase):
c16, c16,
c18, c18,
c19, c19,
c20,
idSeason, idSeason,
userrating) userrating)
VALUES VALUES
(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
''', (args)) ''', (args))
@db.catch_operationalerrors @db.catch_operationalerrors
@ -942,6 +860,7 @@ class KodiVideoDB(common.KodiDBBase):
c16 = ?, c16 = ?,
c18 = ?, c18 = ?,
c19 = ?, c19 = ?,
c20 = ?,
idFile=?, idFile=?,
idSeason = ?, idSeason = ?,
userrating = ? userrating = ?

49
resources/lib/library_sync/common.py
View file

@ -1,40 +1,41 @@
#!/usr/bin/env python #!/usr/bin/env python
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from __future__ import absolute_import, division, unicode_literals from __future__ import absolute_import, division, unicode_literals
from logging import getLogger
import xbmc import xbmc
from .. import utils, app, variables as v from .. import utils, app, variables as v
LOG = getLogger('PLEX.sync')
PLAYLIST_SYNC_ENABLED = (v.DEVICE != 'Microsoft UWP' and PLAYLIST_SYNC_ENABLED = (v.DEVICE != 'Microsoft UWP' and
utils.settings('enablePlaylistSync') == 'true') utils.settings('enablePlaylistSync') == 'true')
class fullsync_mixin(object): class LibrarySyncMixin(object):
def __init__(self): def suspend(self, block=False, timeout=None):
self._canceled = False """
Let's NOT suspend sync threads but immediately terminate them
"""
self.cancel()
def abort(self): def wait_while_suspended(self):
"""Hit method to terminate the thread""" """
self._canceled = True Return immediately
# Let's NOT suspend sync threads but immediately terminate them """
suspend = abort return self.should_cancel()
@property def run(self):
def suspend_reached(self): app.APP.register_thread(self)
"""Since we're not suspending, we'll never set it to True""" LOG.debug('##===--- Starting %s ---===##', self.__class__.__name__)
return False try:
self._run()
@suspend_reached.setter except Exception as err:
def suspend_reached(self): LOG.error('Exception encountered: %s', err)
pass utils.ERROR(notify=True)
finally:
def resume(self): app.APP.deregister_thread(self)
"""Obsolete since we're not suspending""" LOG.debug('##===--- %s Stopped ---===##', self.__class__.__name__)
pass
def isCanceled(self):
"""Check whether we should exit this thread"""
return self._canceled
def update_kodi_library(video=True, music=True): def update_kodi_library(video=True, music=True):

63
resources/lib/library_sync/fanart.py
View file

@ -27,48 +27,51 @@ class FanartThread(backgroundthread.KillableThread):
self.refresh = refresh self.refresh = refresh
super(FanartThread, self).__init__() super(FanartThread, self).__init__()
def isSuspended(self): def should_suspend(self):
return self._suspended or app.APP.is_playing_video return self._suspended or app.APP.is_playing_video
def run(self): def run(self):
LOG.info('Starting FanartThread') LOG.info('Starting FanartThread')
app.APP.register_fanart_thread(self) app.APP.register_fanart_thread(self)
try: try:
self._run_internal() self._run()
except Exception: except Exception:
utils.ERROR(notify=True) utils.ERROR(notify=True)
finally: finally:
app.APP.deregister_fanart_thread(self) app.APP.deregister_fanart_thread(self)
def _run_internal(self): def _loop(self):
for typus in SUPPORTED_TYPES:
offset = 0
while True:
with PlexDB() as plexdb:
# Keep DB connection open only for a short period of time!
if self.refresh:
batch = list(plexdb.every_plex_id(typus,
offset,
BATCH_SIZE))
else:
batch = list(plexdb.missing_fanart(typus,
offset,
BATCH_SIZE))
for plex_id in batch:
# Do the actual, time-consuming processing
if self.should_suspend() or self.should_cancel():
return False
process_fanart(plex_id, typus, self.refresh)
if len(batch) < BATCH_SIZE:
break
offset += BATCH_SIZE
return True
def _run(self):
finished = False finished = False
try: while not finished:
for typus in SUPPORTED_TYPES: finished = self._loop()
offset = 0 if self.wait_while_suspended():
while True: break
with PlexDB() as plexdb: LOG.info('FanartThread finished: %s', finished)
# Keep DB connection open only for a short period of time! self.callback(finished)
if self.refresh:
batch = list(plexdb.every_plex_id(typus,
offset,
BATCH_SIZE))
else:
batch = list(plexdb.missing_fanart(typus,
offset,
BATCH_SIZE))
for plex_id in batch:
# Do the actual, time-consuming processing
if self.wait_while_suspended():
return
process_fanart(plex_id, typus, self.refresh)
if len(batch) < BATCH_SIZE:
break
offset += BATCH_SIZE
else:
finished = True
finally:
LOG.info('FanartThread finished: %s', finished)
self.callback(finished)
class FanartTask(backgroundthread.Task): class FanartTask(backgroundthread.Task):

78
resources/lib/library_sync/fill_metadata_queue.py Normal file
View file

@ -0,0 +1,78 @@
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, unicode_literals
from logging import getLogger
from Queue import Empty
from . import common, sections
from ..plex_db import PlexDB
from .. import backgroundthread
LOG = getLogger('PLEX.sync.fill_metadata_queue')
QUEUE_TIMEOUT = 10 # seconds
class FillMetadataQueue(common.LibrarySyncMixin,
backgroundthread.KillableThread):
"""
Determines which plex_ids we need to sync and puts these ids in a separate
queue. Will use a COPIED plex.db file (plex-copy.db) in order to read much
faster without the writing thread stalling
"""
def __init__(self, repair, section_queue, get_metadata_queue,
processing_queue):
self.repair = repair
self.section_queue = section_queue
self.get_metadata_queue = get_metadata_queue
self.processing_queue = processing_queue
super(FillMetadataQueue, self).__init__()
def _process_section(self, section):
# Initialize only once to avoid loosing the last value before we're
# breaking the for loop
LOG.debug('Process section %s with %s items',
section, section.number_of_items)
count = 0
do_process_section = False
with PlexDB(lock=False, copy=True) as plexdb:
for xml in section.iterator:
if self.should_cancel():
break
plex_id = int(xml.get('ratingKey'))
checksum = int('{}{}'.format(
plex_id,
xml.get('updatedAt',
xml.get('addedAt', '1541572987'))))
if (not self.repair and
plexdb.checksum(plex_id, section.plex_type) == checksum):
continue
try:
self.get_metadata_queue.put((count, plex_id, section),
timeout=QUEUE_TIMEOUT)
except Empty:
LOG.error('Putting %s in get_metadata_queue timed out - '
'aborting sync now', plex_id)
section.sync_successful = False
break
count += 1
if not do_process_section:
do_process_section = True
self.processing_queue.add_section(section)
LOG.debug('Put section in queue with %s items: %s',
section.number_of_items, section)
# We might have received LESS items from the PMS than anticipated.
# Ensures that our queues finish
LOG.debug('%s items to process for section %s', count, section)
section.number_of_items = count
def _run(self):
while not self.should_cancel():
section = self.section_queue.get()
self.section_queue.task_done()
if section is None:
break
self._process_section(section)
# Signal the download metadata threads to stop with a sentinel
self.get_metadata_queue.put(None)
# Sentinel for the process_thread once we added everything else
self.processing_queue.add_sentinel(sections.Section())

493
resources/lib/library_sync/full_sync.py
View file

@ -3,345 +3,208 @@
from __future__ import absolute_import, division, unicode_literals from __future__ import absolute_import, division, unicode_literals
from logging import getLogger from logging import getLogger
import Queue import Queue
import copy
import xbmcgui import xbmcgui
from .get_metadata import GetMetadataTask, reset_collections from .get_metadata import GetMetadataThread
from .fill_metadata_queue import FillMetadataQueue
from .process_metadata import ProcessMetadataThread
from . import common, sections from . import common, sections
from .. import utils, timing, backgroundthread, variables as v, app from .. import utils, timing, backgroundthread as bg, variables as v, app
from .. import plex_functions as PF, itemtypes from .. import plex_functions as PF, itemtypes, path_ops
from ..plex_db import PlexDB
if common.PLAYLIST_SYNC_ENABLED: if common.PLAYLIST_SYNC_ENABLED:
from .. import playlists from .. import playlists
LOG = getLogger('PLEX.sync.full_sync') LOG = getLogger('PLEX.sync.full_sync')
# How many items will be put through the processing chain at once? DELETION_BATCH_SIZE = 250
BATCH_SIZE = 2000 PLAYSTATE_BATCH_SIZE = 5000
# Max. number of plex_ids held in memory for later processing
BACKLOG_QUEUE_SIZE = 10000
# Max number of xmls held in memory
XML_QUEUE_SIZE = 500
# Safety margin to filter PMS items - how many seconds to look into the past? # Safety margin to filter PMS items - how many seconds to look into the past?
UPDATED_AT_SAFETY = 60 * 5 UPDATED_AT_SAFETY = 60 * 5
LAST_VIEWED_AT_SAFETY = 60 * 5 LAST_VIEWED_AT_SAFETY = 60 * 5
class InitNewSection(object): class FullSync(common.LibrarySyncMixin, bg.KillableThread):
"""
Throw this into the queue used for ProcessMetadata to tell it which
Plex library section we're looking at
"""
def __init__(self, context, total_number_of_items, section_name,
section_id, plex_type):
self.context = context
self.total = total_number_of_items
self.name = section_name
self.id = section_id
self.plex_type = plex_type
class FullSync(common.fullsync_mixin):
def __init__(self, repair, callback, show_dialog): def __init__(self, repair, callback, show_dialog):
""" """
repair=True: force sync EVERY item repair=True: force sync EVERY item
""" """
self.successful = True
self.repair = repair self.repair = repair
self.callback = callback self.callback = callback
self.queue = None
self.process_thread = None
self.current_sync = None
self.plexdb = None
self.plex_type = None
self.section_type = None
self.worker_count = int(utils.settings('syncThreadNumber'))
self.item_count = 0
# For progress dialog # For progress dialog
self.show_dialog = show_dialog self.show_dialog = show_dialog
self.show_dialog_userdata = utils.settings('playstate_sync_indicator') == 'true' self.show_dialog_userdata = utils.settings('playstate_sync_indicator') == 'true'
self.dialog = None if self.show_dialog:
self.total = 0 self.dialog = xbmcgui.DialogProgressBG()
self.current = 0 self.dialog.create(utils.lang(39714))
self.processed = 0 else:
self.title = '' self.dialog = None
self.section = None self.current_time = timing.plex_now()
self.section_name = None self.last_section = sections.Section()
self.section_type_text = None
self.context = None
self.get_children = None
self.successful = None
self.section_success = None
self.install_sync_done = utils.settings('SyncInstallRunDone') == 'true' self.install_sync_done = utils.settings('SyncInstallRunDone') == 'true'
self.threader = backgroundthread.ThreaderManager(
worker=backgroundthread.NonstoppingBackgroundWorker,
worker_count=self.worker_count)
super(FullSync, self).__init__() super(FullSync, self).__init__()
def update_progressbar(self): def update_progressbar(self, section, title, current):
if self.dialog: if not self.dialog:
try:
progress = int(float(self.current) / float(self.total) * 100.0)
except ZeroDivisionError:
progress = 0
self.dialog.update(progress,
'%s (%s)' % (self.section_name, self.section_type_text),
'%s %s/%s'
% (self.title, self.current, self.total))
if app.APP.is_playing_video:
self.dialog.close()
self.dialog = None
def process_item(self, xml_item):
"""
Processes a single library item
"""
plex_id = int(xml_item.get('ratingKey'))
if not self.repair and self.plexdb.checksum(plex_id, self.plex_type) == \
int('%s%s' % (plex_id,
xml_item.get('updatedAt',
xml_item.get('addedAt', 1541572987)))):
return return
self.threader.addTask(GetMetadataTask(self.queue, current += 1
plex_id,
self.plex_type,
self.get_children,
self.item_count))
self.item_count += 1
def update_library(self):
LOG.debug('Writing changes to Kodi library now')
i = 0
if not self.section:
_, self.section = self.queue.get()
self.queue.task_done()
while not self.isCanceled() and self.item_count > 0:
section = self.section
if not section:
break
LOG.debug('Start or continue processing section %s (%ss)',
section.name, section.plex_type)
self.processed = 0
self.total = section.total
self.section_name = section.name
self.section_type_text = utils.lang(
v.TRANSLATION_FROM_PLEXTYPE[section.plex_type])
with section.context(self.current_sync) as context:
while not self.isCanceled() and self.item_count > 0:
try:
_, item = self.queue.get(block=False)
except backgroundthread.Queue.Empty:
if self.threader.threader.working():
app.APP.monitor.waitForAbort(0.02)
continue
else:
# Try again, in case a thread just finished
i += 1
if i == 3:
break
continue
i = 0
self.queue.task_done()
if isinstance(item, dict):
context.add_update(item['xml'][0],
section_name=section.name,
section_id=section.id,
children=item['children'])
self.title = item['xml'][0].get('title')
self.processed += 1
elif isinstance(item, InitNewSection) or item is None:
self.section = item
break
else:
raise ValueError('Unknown type %s' % type(item))
self.item_count -= 1
self.current += 1
self.update_progressbar()
if self.processed == 500:
self.processed = 0
context.commit()
LOG.debug('Done writing changes to Kodi library')
@utils.log_time
def addupdate_section(self, section):
LOG.debug('Processing library section for new or changed items %s',
section)
if not self.install_sync_done:
app.SYNC.path_verified = False
try: try:
# Sync new, updated and deleted items progress = int(float(current) / float(section.number_of_items) * 100.0)
iterator = section.iterator except ZeroDivisionError:
# Tell the processing thread about this new section progress = 0
queue_info = InitNewSection(section.context, self.dialog.update(progress,
iterator.total, '%s (%s)' % (section.name, section.section_type_text),
iterator.get('librarySectionTitle', '%s %s/%s'
iterator.get('title1')), % (title, current, section.number_of_items))
section.section_id, if app.APP.is_playing_video:
section.plex_type) self.dialog.close()
self.queue.put((-1, queue_info)) self.dialog = None
last = True
# To keep track of the item-number in order to kill while loops @staticmethod
self.item_count = 0 def copy_plex_db():
self.current = 0 """
# Initialize only once to avoid loosing the last value before Takes the current plex.db file and copies it to plex-copy.db
# we're breaking the for loop This will allow us to have "concurrent" connections during adding/
loop = common.tag_last(iterator) updating items, increasing sync speed tremendously.
while True: Using the same DB with e.g. WAL mode did not really work out...
# Check Plex DB to see what we need to add/update """
with PlexDB() as self.plexdb: path_ops.copyfile(v.DB_PLEX_PATH, v.DB_PLEX_COPY_PATH)
for last, xml_item in loop:
if self.isCanceled():
return False
self.process_item(xml_item)
if self.item_count == BATCH_SIZE:
break
# Make sure Plex DB above is closed before adding/updating!
self.update_library()
if last:
break
reset_collections()
return True
except RuntimeError:
LOG.error('Could not entirely process section %s', section)
return False
@utils.log_time @utils.log_time
def process_new_and_changed_items(self, section_queue, processing_queue):
LOG.debug('Start working')
get_metadata_queue = Queue.Queue(maxsize=BACKLOG_QUEUE_SIZE)
scanner_thread = FillMetadataQueue(self.repair,
section_queue,
get_metadata_queue,
processing_queue)
scanner_thread.start()
metadata_threads = [
GetMetadataThread(get_metadata_queue, processing_queue)
for _ in range(int(utils.settings('syncThreadNumber')))
]
for t in metadata_threads:
t.start()
process_thread = ProcessMetadataThread(self.current_time,
processing_queue,
self.update_progressbar)
process_thread.start()
LOG.debug('Waiting for scanner thread to finish up')
scanner_thread.join()
LOG.debug('Waiting for metadata download threads to finish up')
for t in metadata_threads:
t.join()
LOG.debug('Download metadata threads finished')
process_thread.join()
self.successful = process_thread.successful
LOG.debug('threads finished work. successful: %s', self.successful)
@utils.log_time
def processing_loop_playstates(self, section_queue):
while not self.should_cancel():
section = section_queue.get()
section_queue.task_done()
if section is None:
break
self.playstate_per_section(section)
def playstate_per_section(self, section): def playstate_per_section(self, section):
LOG.debug('Processing %s playstates for library section %s', LOG.debug('Processing %s playstates for library section %s',
section.iterator.total, section) section.number_of_items, section)
try: try:
# Sync new, updated and deleted items with section.context(self.current_time) as context:
iterator = section.iterator for xml in section.iterator:
# Tell the processing thread about this new section section.count += 1
queue_info = InitNewSection(section.context, if not context.update_userdata(xml, section.plex_type):
iterator.total, # Somehow did not sync this item yet
section.name, context.add_update(xml,
section.section_id, section_name=section.name,
section.plex_type) section_id=section.section_id)
self.queue.put((-1, queue_info)) context.plexdb.update_last_sync(int(xml.attrib['ratingKey']),
self.total = iterator.total section.plex_type,
self.section_name = section.name self.current_time)
self.section_type_text = utils.lang( self.update_progressbar(section, '', section.count - 1)
v.TRANSLATION_FROM_PLEXTYPE[section.plex_type]) if section.count % PLAYSTATE_BATCH_SIZE == 0:
self.current = 0 context.commit()
last = True
loop = common.tag_last(iterator)
while True:
with section.context(self.current_sync) as itemtype:
for i, (last, xml_item) in enumerate(loop):
if self.isCanceled():
return False
if not itemtype.update_userdata(xml_item, section.plex_type):
# Somehow did not sync this item yet
itemtype.add_update(xml_item,
section_name=section.name,
section_id=section.section_id)
itemtype.plexdb.update_last_sync(int(xml_item.attrib['ratingKey']),
section.plex_type,
self.current_sync)
self.current += 1
self.update_progressbar()
if (i + 1) % (10 * BATCH_SIZE) == 0:
break
if last:
break
return True
except RuntimeError: except RuntimeError:
LOG.error('Could not entirely process section %s', section) LOG.error('Could not entirely process section %s', section)
return False self.successful = False
def threaded_get_iterators(self, kinds, queue, all_items=False): def threaded_get_generators(self, kinds, section_queue, all_items):
""" """
Getting iterators is costly, so let's do it asynchronously Getting iterators is costly, so let's do it in a dedicated thread
""" """
LOG.debug('Start threaded_get_generators')
try: try:
for kind in kinds: for kind in kinds:
for section in (x for x in app.SYNC.sections for section in (x for x in app.SYNC.sections
if x.section_type == kind[1]): if x.section_type == kind[1]):
if self.isCanceled(): if self.should_cancel():
LOG.debug('Need to exit now') LOG.debug('Need to exit now')
return return
if not section.sync_to_kodi: if not section.sync_to_kodi:
LOG.info('User chose to not sync section %s', section) LOG.info('User chose to not sync section %s', section)
continue continue
element = copy.deepcopy(section) section = sections.get_sync_section(section,
element.plex_type = kind[0] plex_type=kind[0])
element.section_type = element.plex_type
element.context = kind[2]
element.get_children = kind[3]
element.Queue = kind[4]
if self.repair or all_items: if self.repair or all_items:
updated_at = None updated_at = None
else: else:
updated_at = section.last_sync - UPDATED_AT_SAFETY \ updated_at = section.last_sync - UPDATED_AT_SAFETY \
if section.last_sync else None if section.last_sync else None
try: try:
element.iterator = PF.get_section_iterator( section.iterator = PF.get_section_iterator(
section.section_id, section.section_id,
plex_type=element.plex_type, plex_type=section.plex_type,
updated_at=updated_at, updated_at=updated_at,
last_viewed_at=None) last_viewed_at=None)
except RuntimeError: except RuntimeError:
LOG.warn('Sync at least partially unsuccessful') LOG.error('Sync at least partially unsuccessful!')
self.successful = False LOG.error('Error getting section iterator %s', section)
self.section_success = False
else: else:
queue.put(element) section.number_of_items = section.iterator.total
if section.number_of_items > 0:
section_queue.put(section)
LOG.debug('Put section in queue with %s items: %s',
section.number_of_items, section)
except Exception: except Exception:
utils.ERROR(notify=True) utils.ERROR(notify=True)
finally: finally:
queue.put(None) # Sentinel for the section queue
section_queue.put(None)
LOG.debug('Exiting threaded_get_generators')
def full_library_sync(self): def full_library_sync(self):
""" section_queue = Queue.Queue()
""" processing_queue = bg.ProcessingQueue(maxsize=XML_QUEUE_SIZE)
# structure:
# (plex_type,
# section_type,
# context for itemtype,
# download children items, e.g. songs for a specific album?,
# Queue)
kinds = [ kinds = [
(v.PLEX_TYPE_MOVIE, v.PLEX_TYPE_MOVIE, itemtypes.Movie, False, Queue.Queue), (v.PLEX_TYPE_MOVIE, v.PLEX_TYPE_MOVIE),
(v.PLEX_TYPE_SHOW, v.PLEX_TYPE_SHOW, itemtypes.Show, False, Queue.Queue), (v.PLEX_TYPE_SHOW, v.PLEX_TYPE_SHOW),
(v.PLEX_TYPE_SEASON, v.PLEX_TYPE_SHOW, itemtypes.Season, False, Queue.Queue), (v.PLEX_TYPE_SEASON, v.PLEX_TYPE_SHOW),
(v.PLEX_TYPE_EPISODE, v.PLEX_TYPE_SHOW, itemtypes.Episode, False, Queue.Queue) (v.PLEX_TYPE_EPISODE, v.PLEX_TYPE_SHOW)
] ]
if app.SYNC.enable_music: if app.SYNC.enable_music:
kinds.extend([ kinds.extend([
(v.PLEX_TYPE_ARTIST, v.PLEX_TYPE_ARTIST, itemtypes.Artist, False, Queue.Queue), (v.PLEX_TYPE_ARTIST, v.PLEX_TYPE_ARTIST),
(v.PLEX_TYPE_ALBUM, v.PLEX_TYPE_ARTIST, itemtypes.Album, True, backgroundthread.OrderedQueue), (v.PLEX_TYPE_ALBUM, v.PLEX_TYPE_ARTIST),
]) ])
# ADD NEW ITEMS # ADD NEW ITEMS
# Already start setting up the iterators. We need to enforce # We need to enforce syncing e.g. show before season before episode
# syncing e.g. show before season before episode bg.FunctionAsTask(self.threaded_get_generators,
iterator_queue = Queue.Queue() None,
task = backgroundthread.FunctionAsTask(self.threaded_get_iterators, kinds, section_queue, False).start()
None, # Do the heavy lifting
kinds, self.process_new_and_changed_items(section_queue, processing_queue)
iterator_queue)
backgroundthread.BGThreader.addTask(task)
while True:
self.section_success = True
section = iterator_queue.get()
iterator_queue.task_done()
if section is None:
break
# Setup our variables
self.plex_type = section.plex_type
self.section_type = section.section_type
self.context = section.context
self.get_children = section.get_children
self.queue = section.Queue()
# Now do the heavy lifting
if self.isCanceled() or not self.addupdate_section(section):
return False
if self.section_success:
# Need to check because a thread might have missed to get
# some items from the PMS
with PlexDB() as plexdb:
# Set the new time mark for the next delta sync
plexdb.update_section_last_sync(section.section_id,
self.current_sync)
common.update_kodi_library(video=True, music=True) common.update_kodi_library(video=True, music=True)
if self.should_cancel() or not self.successful:
return
# Sync Plex playlists to Kodi and vice-versa # Sync Plex playlists to Kodi and vice-versa
if common.PLAYLIST_SYNC_ENABLED: if common.PLAYLIST_SYNC_ENABLED:
@ -351,48 +214,29 @@ class FullSync(common.fullsync_mixin):
self.dialog = xbmcgui.DialogProgressBG() self.dialog = xbmcgui.DialogProgressBG()
# "Synching playlists" # "Synching playlists"
self.dialog.create(utils.lang(39715)) self.dialog.create(utils.lang(39715))
if not playlists.full_sync(): if not playlists.full_sync() or self.should_cancel():
return False return
# SYNC PLAYSTATE of ALL items (otherwise we won't pick up on items that # SYNC PLAYSTATE of ALL items (otherwise we won't pick up on items that
# were set to unwatched). Also mark all items on the PMS to be able # were set to unwatched). Also mark all items on the PMS to be able
# to delete the ones still in Kodi # to delete the ones still in Kodi
LOG.info('Start synching playstate and userdata for every item') LOG.debug('Start synching playstate and userdata for every item')
# In order to not delete all your songs again
if app.SYNC.enable_music: if app.SYNC.enable_music:
# We don't need to enforce the album order now # In order to not delete all your songs again
kinds.pop(5)
kinds.extend([ kinds.extend([
(v.PLEX_TYPE_ALBUM, v.PLEX_TYPE_ARTIST, itemtypes.Album, True, Queue.Queue), (v.PLEX_TYPE_SONG, v.PLEX_TYPE_ARTIST),
(v.PLEX_TYPE_SONG, v.PLEX_TYPE_ARTIST, itemtypes.Song, True, Queue.Queue),
]) ])
# Make sure we're not showing an item's title in the sync dialog # Make sure we're not showing an item's title in the sync dialog
self.title = ''
self.threader.shutdown()
self.threader = None
if not self.show_dialog_userdata and self.dialog: if not self.show_dialog_userdata and self.dialog:
# Close the progress indicator dialog # Close the progress indicator dialog
self.dialog.close() self.dialog.close()
self.dialog = None self.dialog = None
task = backgroundthread.FunctionAsTask(self.threaded_get_iterators, bg.FunctionAsTask(self.threaded_get_generators,
None, None,
kinds, kinds, section_queue, True).start()
iterator_queue, self.processing_loop_playstates(section_queue)
all_items=True) if self.should_cancel() or not self.successful:
backgroundthread.BGThreader.addTask(task) return
while True:
section = iterator_queue.get()
iterator_queue.task_done()
if section is None:
break
# Setup our variables
self.plex_type = section.plex_type
self.section_type = section.section_type
self.context = section.context
self.get_children = section.get_children
# Now do the heavy lifting
if self.isCanceled() or not self.playstate_per_section(section):
return False
# Delete movies that are not on Plex anymore # Delete movies that are not on Plex anymore
LOG.debug('Looking for items to delete') LOG.debug('Looking for items to delete')
@ -411,61 +255,40 @@ class FullSync(common.fullsync_mixin):
for plex_type, context in kinds: for plex_type, context in kinds:
# Delete movies that are not on Plex anymore # Delete movies that are not on Plex anymore
while True: while True:
with context(self.current_sync) as ctx: with context(self.current_time) as ctx:
plex_ids = list(ctx.plexdb.plex_id_by_last_sync(plex_type, plex_ids = list(
self.current_sync, ctx.plexdb.plex_id_by_last_sync(plex_type,
BATCH_SIZE)) self.current_time,
DELETION_BATCH_SIZE))
for plex_id in plex_ids: for plex_id in plex_ids:
if self.isCanceled(): if self.should_cancel():
return False return
ctx.remove(plex_id, plex_type) ctx.remove(plex_id, plex_type)
if len(plex_ids) < BATCH_SIZE: if len(plex_ids) < DELETION_BATCH_SIZE:
break break
LOG.debug('Done deleting') LOG.debug('Done looking for items to delete')
return True
def run(self):
app.APP.register_thread(self)
try:
self._run()
finally:
app.APP.deregister_thread(self)
LOG.info('Done full_sync')
@utils.log_time @utils.log_time
def _run(self): def _run(self):
self.current_sync = timing.plex_now()
# Get latest Plex libraries and build playlist and video node files
if self.isCanceled() or not sections.sync_from_pms(self):
return
self.successful = True
try: try:
if self.show_dialog: # Get latest Plex libraries and build playlist and video node files
self.dialog = xbmcgui.DialogProgressBG() if self.should_cancel() or not sections.sync_from_pms(self):
self.dialog.create(utils.lang(39714))
# Actual syncing - do only new items first
LOG.info('Running full_library_sync with repair=%s',
self.repair)
if self.isCanceled() or not self.full_library_sync():
self.successful = False
return return
self.copy_plex_db()
self.full_library_sync()
finally: finally:
common.update_kodi_library(video=True, music=True) common.update_kodi_library(video=True, music=True)
if self.dialog: if self.dialog:
self.dialog.close() self.dialog.close()
if self.threader: if not self.successful and not self.should_cancel():
self.threader.shutdown()
self.threader = None
if not self.successful and not self.isCanceled():
# "ERROR in library sync" # "ERROR in library sync"
utils.dialog('notification', utils.dialog('notification',
heading='{plex}', heading='{plex}',
message=utils.lang(39410), message=utils.lang(39410),
icon='{error}') icon='{error}')
if self.callback: self.callback(self.successful)
self.callback(self.successful)
def start(show_dialog, repair=False, callback=None): def start(show_dialog, repair=False, callback=None):
# Call run() and NOT start in order to not spawn another thread
FullSync(repair, callback, show_dialog).run() FullSync(repair, callback, show_dialog).run()

168
resources/lib/library_sync/get_metadata.py
View file

@ -4,66 +4,43 @@ from logging import getLogger
from . import common from . import common
from ..plex_api import API from ..plex_api import API
from .. import plex_functions as PF, backgroundthread, utils, variables as v from .. import backgroundthread, plex_functions as PF, utils, variables as v
LOG = getLogger("PLEX." + __name__)
LOG = getLogger('PLEX.sync.get_metadata')
LOCK = backgroundthread.threading.Lock() LOCK = backgroundthread.threading.Lock()
# List of tuples: (collection index [as in an item's metadata with "Collection
# id"], collection plex id)
COLLECTION_MATCH = None
# Dict with entries of the form <collection index>: <collection xml>
COLLECTION_XMLS = {}
def reset_collections(): class GetMetadataThread(common.LibrarySyncMixin,
""" backgroundthread.KillableThread):
Collections seem unique to Plex sections
"""
global LOCK, COLLECTION_MATCH, COLLECTION_XMLS
with LOCK:
COLLECTION_MATCH = None
COLLECTION_XMLS = {}
class GetMetadataTask(common.fullsync_mixin, backgroundthread.Task):
""" """
Threaded download of Plex XML metadata for a certain library item. Threaded download of Plex XML metadata for a certain library item.
Fills the queue with the downloaded etree XML objects Fills the queue with the downloaded etree XML objects
Input:
queue Queue.Queue() object where this thread will store
the downloaded metadata XMLs as etree objects
""" """
def __init__(self, queue, plex_id, plex_type, get_children=False, def __init__(self, get_metadata_queue, processing_queue):
count=None): self.get_metadata_queue = get_metadata_queue
self.queue = queue self.processing_queue = processing_queue
self.plex_id = plex_id super(GetMetadataThread, self).__init__()
self.plex_type = plex_type
self.get_children = get_children
self.count = count
super(GetMetadataTask, self).__init__()
def _collections(self, item): def _collections(self, item):
global COLLECTION_MATCH, COLLECTION_XMLS
api = API(item['xml'][0]) api = API(item['xml'][0])
if COLLECTION_MATCH is None: collection_match = item['section'].collection_match
COLLECTION_MATCH = PF.collections(api.library_section_id()) collection_xmls = item['section'].collection_xmls
if COLLECTION_MATCH is None: if collection_match is None:
collection_match = PF.collections(api.library_section_id())
if collection_match is None:
LOG.error('Could not download collections') LOG.error('Could not download collections')
return return
# Extract what we need to know # Extract what we need to know
COLLECTION_MATCH = \ collection_match = \
[(utils.cast(int, x.get('index')), [(utils.cast(int, x.get('index')),
utils.cast(int, x.get('ratingKey'))) for x in COLLECTION_MATCH] utils.cast(int, x.get('ratingKey'))) for x in collection_match]
item['children'] = {} item['children'] = {}
for plex_set_id, set_name in api.collections(): for plex_set_id, set_name in api.collections():
if self.isCanceled(): if self.should_cancel():
return return
if plex_set_id not in COLLECTION_XMLS: if plex_set_id not in collection_xmls:
# Get Plex metadata for collections - a pain # Get Plex metadata for collections - a pain
for index, collection_plex_id in COLLECTION_MATCH: for index, collection_plex_id in collection_match:
if index == plex_set_id: if index == plex_set_id:
collection_xml = PF.GetPlexMetadata(collection_plex_id) collection_xml = PF.GetPlexMetadata(collection_plex_id)
try: try:
@ -72,54 +49,75 @@ class GetMetadataTask(common.fullsync_mixin, backgroundthread.Task):
LOG.error('Could not get collection %s %s', LOG.error('Could not get collection %s %s',
collection_plex_id, set_name) collection_plex_id, set_name)
continue continue
COLLECTION_XMLS[plex_set_id] = collection_xml collection_xmls[plex_set_id] = collection_xml
break break
else: else:
LOG.error('Did not find Plex collection %s %s', LOG.error('Did not find Plex collection %s %s',
plex_set_id, set_name) plex_set_id, set_name)
continue continue
item['children'][plex_set_id] = COLLECTION_XMLS[plex_set_id] item['children'][plex_set_id] = collection_xmls[plex_set_id]
def run(self): def _process_abort(self, count, section):
""" # Make sure other threads will also receive sentinel
Do the work self.get_metadata_queue.put(None)
""" if count is not None:
if self.isCanceled(): self._process_skipped_item(count, section)
return
# Download Metadata def _process_skipped_item(self, count, section):
item = { section.sync_successful = False
'xml': PF.GetPlexMetadata(self.plex_id), # Add a "dummy" item so we're not skipping a beat
'children': None self.processing_queue.put((count, {'section': section, 'xml': None}))
}
if item['xml'] is None: def _run(self):
# Did not receive a valid XML - skip that item for now while True:
LOG.error("Could not get metadata for %s. Skipping that item " item = self.get_metadata_queue.get()
"for now", self.plex_id)
return
elif item['xml'] == 401:
LOG.error('HTTP 401 returned by PMS. Too much strain? '
'Cancelling sync for now')
utils.window('plex_scancrashed', value='401')
return
if not self.isCanceled() and self.plex_type == v.PLEX_TYPE_MOVIE:
# Check for collections/sets
collections = False
for child in item['xml'][0]:
if child.tag == 'Collection':
collections = True
break
if collections:
global LOCK
with LOCK:
self._collections(item)
if not self.isCanceled() and self.get_children:
children_xml = PF.GetAllPlexChildren(self.plex_id)
try: try:
children_xml[0].attrib if item is None or self.should_cancel():
except (TypeError, IndexError, AttributeError): self._process_abort(item[0] if item else None,
LOG.error('Could not get children for Plex id %s', item[2] if item else None)
self.plex_id) break
else: count, plex_id, section = item
item['children'] = children_xml item = {
if not self.isCanceled(): 'xml': PF.GetPlexMetadata(plex_id), # This will block
self.queue.put((self.count, item)) 'children': None,
'section': section
}
if item['xml'] is None:
# Did not receive a valid XML - skip that item for now
LOG.error("Could not get metadata for %s. Skipping item "
"for now", plex_id)
self._process_skipped_item(count, section)
continue
elif item['xml'] == 401:
LOG.error('HTTP 401 returned by PMS. Too much strain? '
'Cancelling sync for now')
utils.window('plex_scancrashed', value='401')
self._process_abort(count, section)
break
if section.plex_type == v.PLEX_TYPE_MOVIE:
# Check for collections/sets
collections = False
for child in item['xml'][0]:
if child.tag == 'Collection':
collections = True
break
if collections:
with LOCK:
self._collections(item)
if section.get_children:
if self.should_cancel():
self._process_abort(count, section)
break
children_xml = PF.GetAllPlexChildren(plex_id) # Will block
try:
children_xml[0].attrib
except (TypeError, IndexError, AttributeError):
LOG.error('Could not get children for Plex id %s',
plex_id)
self._process_skipped_item(count, section)
continue
else:
item['children'] = children_xml
self.processing_queue.put((count, item))
finally:
self.get_metadata_queue.task_done()

92
resources/lib/library_sync/process_metadata.py Normal file
View file

@ -0,0 +1,92 @@
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, unicode_literals
from logging import getLogger
from . import common, sections
from ..plex_db import PlexDB
from .. import backgroundthread, app
LOG = getLogger('PLEX.sync.process_metadata')
COMMIT_TO_DB_EVERY_X_ITEMS = 500
class ProcessMetadataThread(common.LibrarySyncMixin,
backgroundthread.KillableThread):
"""
Invoke once in order to process the received PMS metadata xmls
"""
def __init__(self, current_time, processing_queue, update_progressbar):
self.current_time = current_time
self.processing_queue = processing_queue
self.update_progressbar = update_progressbar
self.last_section = sections.Section()
self.successful = True
super(ProcessMetadataThread, self).__init__()
def start_section(self, section):
if section != self.last_section:
if self.last_section:
self.finish_last_section()
LOG.debug('Start or continue processing section %s', section)
self.last_section = section
# Warn the user for this new section if we cannot access a file
app.SYNC.path_verified = False
else:
LOG.debug('Resume processing section %s', section)
def finish_last_section(self):
if (not self.should_cancel() and self.last_section and
self.last_section.sync_successful):
# Check for should_cancel() because we cannot be sure that we
# processed every item of the section
with PlexDB() as plexdb:
# Set the new time mark for the next delta sync
plexdb.update_section_last_sync(self.last_section.section_id,
self.current_time)
LOG.info('Finished processing section successfully: %s',
self.last_section)
elif self.last_section and not self.last_section.sync_successful:
LOG.warn('Sync not successful for section %s', self.last_section)
self.successful = False
def _get(self):
item = {'xml': None}
while item and item['xml'] is None:
item = self.processing_queue.get()
self.processing_queue.task_done()
return item
def _run(self):
# There are 2 sentinels: None for aborting/ending this thread, the dict
# {'section': section, 'xml': None} for skipped/invalid items
item = self._get()
if item:
section = item['section']
processed = 0
self.start_section(section)
while not self.should_cancel():
if item is None:
break
elif item['section'] != section:
# We received an entirely new section
self.start_section(item['section'])
section = item['section']
with section.context(self.current_time) as context:
while not self.should_cancel():
if item is None or item['section'] != section:
break
self.update_progressbar(section,
item['xml'][0].get('title'),
section.count)
context.add_update(item['xml'][0],
section_name=section.name,
section_id=section.section_id,
children=item['children'])
processed += 1
section.count += 1
if processed == COMMIT_TO_DB_EVERY_X_ITEMS:
processed = 0
context.commit()
item = self._get()
self.finish_last_section()

67
resources/lib/library_sync/sections.py
View file

@ -16,7 +16,7 @@ LOG = getLogger('PLEX.sync.sections')
BATCH_SIZE = 500 BATCH_SIZE = 500
# Need a way to interrupt our synching process # Need a way to interrupt our synching process
IS_CANCELED = None SHOULD_CANCEL = None
LIBRARY_PATH = path_ops.translate_path('special://profile/library/video/') LIBRARY_PATH = path_ops.translate_path('special://profile/library/video/')
# The video library might not yet exist for this user - create it # The video library might not yet exist for this user - create it
@ -54,6 +54,8 @@ class Section(object):
self.content = None # unicode self.content = None # unicode
# Setting the section_type WILL re_set sync_to_kodi! # Setting the section_type WILL re_set sync_to_kodi!
self._section_type = None # unicode self._section_type = None # unicode
# E.g. "season" or "movie" (translated)
self.section_type_text = None
# Do we sync all items of this section to the Kodi DB? # Do we sync all items of this section to the Kodi DB?
# This will be set with section_type!! # This will be set with section_type!!
self.sync_to_kodi = None # bool self.sync_to_kodi = None # bool
@ -77,13 +79,9 @@ class Section(object):
self.order = None self.order = None
# Original PMS xml for this section, including children # Original PMS xml for this section, including children
self.xml = None self.xml = None
# Attributes that will be initialized later by full_sync.py
self.iterator = None
self.context = None
self.get_children = None
# A section_type encompasses possible several plex_types! E.g. shows # A section_type encompasses possible several plex_types! E.g. shows
# contain shows, seasons, episodes # contain shows, seasons, episodes
self.plex_type = None self._plex_type = None
if xml_element is not None: if xml_element is not None:
self.from_xml(xml_element) self.from_xml(xml_element)
elif section_db_element: elif section_db_element:
@ -106,9 +104,14 @@ class Section(object):
self.section_type is not None) self.section_type is not None)
def __eq__(self, section): def __eq__(self, section):
"""
Sections compare equal if their section_id, name and plex_type (first
prio) OR section_type (if there is no plex_type is set) compare equal
"""
return (self.section_id == section.section_id and return (self.section_id == section.section_id and
self.name == section.name and self.name == section.name and
self.section_type == section.section_type) (self.plex_type == section.plex_type if self.plex_type else
self.section_type == section.section_type))
def __ne__(self, section): def __ne__(self, section):
return not self == section return not self == section
@ -140,6 +143,15 @@ class Section(object):
else: else:
self.sync_to_kodi = True self.sync_to_kodi = True
@property
def plex_type(self):
return self._plex_type
@plex_type.setter
def plex_type(self, value):
self._plex_type = value
self.section_type_text = utils.lang(v.TRANSLATION_FROM_PLEXTYPE[value])
@property @property
def index(self): def index(self):
return self._index return self._index
@ -431,6 +443,39 @@ class Section(object):
self.remove_from_plex() self.remove_from_plex()
def _get_children(plex_type):
if plex_type == v.PLEX_TYPE_ALBUM:
return True
else:
return False
def get_sync_section(section, plex_type):
"""
Deep-copies section and adds certain arguments in order to prep section
for the library sync
"""
section = copy.deepcopy(section)
section.plex_type = plex_type
section.context = itemtypes.ITEMTYPE_FROM_PLEXTYPE[plex_type]
section.get_children = _get_children(plex_type)
# Some more init stuff
# Has sync for this section been successful?
section.sync_successful = True
# List of tuples: (collection index [as in an item's metadata with
# "Collection id"], collection plex id)
section.collection_match = None
# Dict with entries of the form <collection index>: <collection xml>
section.collection_xmls = {}
# Keep count during sync
section.count = 0
# Total number of items that we need to sync
section.number_of_items = 0
# Iterator to get one sync item after the other
section.iterator = None
return section
def force_full_sync(): def force_full_sync():
""" """
Resets the sync timestamp for all sections to 0, thus forcing a subsequent Resets the sync timestamp for all sections to 0, thus forcing a subsequent
@ -490,7 +535,7 @@ def _delete_kodi_db_items(section):
with kodi_context(texture_db=True) as kodidb: with kodi_context(texture_db=True) as kodidb:
typus = context(None, plexdb=plexdb, kodidb=kodidb) typus = context(None, plexdb=plexdb, kodidb=kodidb)
for plex_id in plex_ids: for plex_id in plex_ids:
if IS_CANCELED(): if SHOULD_CANCEL():
return False return False
typus.remove(plex_id) typus.remove(plex_id)
if len(plex_ids) < BATCH_SIZE: if len(plex_ids) < BATCH_SIZE:
@ -582,13 +627,13 @@ def sync_from_pms(parent_self, pick_libraries=False):
pick_libraries=True will prompt the user the select the libraries he pick_libraries=True will prompt the user the select the libraries he
wants to sync wants to sync
""" """
global IS_CANCELED global SHOULD_CANCEL
LOG.info('Starting synching sections from the PMS') LOG.info('Starting synching sections from the PMS')
IS_CANCELED = parent_self.isCanceled SHOULD_CANCEL = parent_self.should_cancel
try: try:
return _sync_from_pms(pick_libraries) return _sync_from_pms(pick_libraries)
finally: finally:
IS_CANCELED = None SHOULD_CANCEL = None
LOG.info('Done synching sections from the PMS: %s', app.SYNC.sections) LOG.info('Done synching sections from the PMS: %s', app.SYNC.sections)

2
resources/lib/library_sync/websocket.py
View file

@ -125,7 +125,7 @@ def process_new_item_message(message):
with itemtypes.ITEMTYPE_FROM_PLEXTYPE[plex_type](timing.unix_timestamp()) as typus: with itemtypes.ITEMTYPE_FROM_PLEXTYPE[plex_type](timing.unix_timestamp()) as typus:
typus.add_update(xml[0], typus.add_update(xml[0],
section_name=xml.get('librarySectionTitle'), section_name=xml.get('librarySectionTitle'),
section_id=xml.get('librarySectionID')) section_id=utils.cast(int, xml.get('librarySectionID')))
cache_artwork(message['plex_id'], plex_type) cache_artwork(message['plex_id'], plex_type)
return True, plex_type in v.PLEX_VIDEOTYPES, plex_type in v.PLEX_AUDIOTYPES return True, plex_type in v.PLEX_VIDEOTYPES, plex_type in v.PLEX_AUDIOTYPES

3
resources/lib/migration.py
View file

@ -64,7 +64,8 @@ def check_migration():
if not utils.compare_version(last_migration, '2.9.3'): if not utils.compare_version(last_migration, '2.9.3'):
LOG.info('Migrating to version 2.9.2') LOG.info('Migrating to version 2.9.2')
# Re-sync all playlists to Kodi # Re-sync all playlists to Kodi
utils.wipe_synched_playlists() from .playlists import remove_synced_playlists
remove_synced_playlists()
if not utils.compare_version(last_migration, '2.9.7'): if not utils.compare_version(last_migration, '2.9.7'):
LOG.info('Migrating to version 2.9.6') LOG.info('Migrating to version 2.9.6')

51
resources/lib/playlists/__init__.py
View file

@ -13,6 +13,7 @@
""" """
from __future__ import absolute_import, division, unicode_literals from __future__ import absolute_import, division, unicode_literals
from logging import getLogger from logging import getLogger
from sqlite3 import OperationalError
from .common import Playlist, PlaylistError, PlaylistObserver, \ from .common import Playlist, PlaylistError, PlaylistObserver, \
kodi_playlist_hash kodi_playlist_hash
@ -38,7 +39,7 @@ SUPPORTED_FILETYPES = (
############################################################################### ###############################################################################
def isCanceled(): def should_cancel():
return app.APP.stop_pkc or app.SYNC.stop_sync return app.APP.stop_pkc or app.SYNC.stop_sync
@ -68,6 +69,22 @@ def kodi_playlist_monitor():
return observer return observer
def remove_synced_playlists():
"""
Deletes all synched playlists on the Kodi side, not on the Plex side
"""
LOG.info('Removing all playlists that we synced to Kodi')
with app.APP.lock_playlists:
try:
paths = db.get_all_kodi_playlist_paths()
except OperationalError:
LOG.info('Playlists table has not yet been set-up')
return
kodi_pl.delete_kodi_playlists(paths)
db.wipe_table()
LOG.info('Done removing all synced playlists')
def websocket(plex_id, status): def websocket(plex_id, status):
""" """
Call this function to process websocket messages from the PMS Call this function to process websocket messages from the PMS
@ -167,7 +184,7 @@ def _full_sync():
# before. If yes, make sure that hashes are identical. If not, sync it. # before. If yes, make sure that hashes are identical. If not, sync it.
old_plex_ids = db.plex_playlist_ids() old_plex_ids = db.plex_playlist_ids()
for xml_playlist in xml: for xml_playlist in xml:
if isCanceled(): if should_cancel():
return False return False
api = API(xml_playlist) api = API(xml_playlist)
try: try:
@ -199,7 +216,7 @@ def _full_sync():
LOG.info('Could not recreate playlist %s', api.plex_id) LOG.info('Could not recreate playlist %s', api.plex_id)
# Get rid of old Plex playlists that were deleted on the Plex side # Get rid of old Plex playlists that were deleted on the Plex side
for plex_id in old_plex_ids: for plex_id in old_plex_ids:
if isCanceled(): if should_cancel():
return False return False
playlist = db.get_playlist(plex_id=plex_id) playlist = db.get_playlist(plex_id=plex_id)
LOG.debug('Removing outdated Plex playlist from Kodi: %s', playlist) LOG.debug('Removing outdated Plex playlist from Kodi: %s', playlist)
@ -213,7 +230,7 @@ def _full_sync():
old_kodi_paths = db.kodi_playlist_paths() old_kodi_paths = db.kodi_playlist_paths()
for root, _, files in path_ops.walk(v.PLAYLIST_PATH): for root, _, files in path_ops.walk(v.PLAYLIST_PATH):
for f in files: for f in files:
if isCanceled(): if should_cancel():
return False return False
path = path_ops.path.join(root, f) path = path_ops.path.join(root, f)
try: try:
@ -244,7 +261,7 @@ def _full_sync():
except PlaylistError: except PlaylistError:
LOG.info('Skipping Kodi playlist %s', path) LOG.info('Skipping Kodi playlist %s', path)
for kodi_path in old_kodi_paths: for kodi_path in old_kodi_paths:
if isCanceled(): if should_cancel():
return False return False
playlist = db.get_playlist(path=kodi_path) playlist = db.get_playlist(path=kodi_path)
if not playlist: if not playlist:
@ -370,19 +387,19 @@ class PlaylistEventhandler(events.FileSystemEventHandler):
""" """
path = event.dest_path if event.event_type == events.EVENT_TYPE_MOVED \ path = event.dest_path if event.event_type == events.EVENT_TYPE_MOVED \
else event.src_path else event.src_path
if not sync_kodi_playlist(path):
return
if path in kodi_pl.IGNORE_KODI_PLAYLIST_CHANGE:
LOG.debug('Ignoring event %s', event)
kodi_pl.IGNORE_KODI_PLAYLIST_CHANGE.remove(path)
return
_method_map = {
events.EVENT_TYPE_MODIFIED: self.on_modified,
events.EVENT_TYPE_MOVED: self.on_moved,
events.EVENT_TYPE_CREATED: self.on_created,
events.EVENT_TYPE_DELETED: self.on_deleted,
}
with app.APP.lock_playlists: with app.APP.lock_playlists:
if not sync_kodi_playlist(path):
return
if path in kodi_pl.IGNORE_KODI_PLAYLIST_CHANGE:
LOG.debug('Ignoring event %s', event)
kodi_pl.IGNORE_KODI_PLAYLIST_CHANGE.remove(path)
return
_method_map = {
events.EVENT_TYPE_MODIFIED: self.on_modified,
events.EVENT_TYPE_MOVED: self.on_moved,
events.EVENT_TYPE_CREATED: self.on_created,
events.EVENT_TYPE_DELETED: self.on_deleted,
}
_method_map[event.event_type](event) _method_map[event.event_type](event)
def on_created(self, event): def on_created(self, event):

17
resources/lib/playlists/db.py
View file

@ -57,6 +57,23 @@ def get_playlist(path=None, plex_id=None):
return playlist return playlist
def get_all_kodi_playlist_paths():
"""
Returns a list with all paths for the playlists on the Kodi side
"""
with PlexDB() as plexdb:
paths = list(plexdb.all_kodi_paths())
return paths
def wipe_table():
"""
Deletes all playlists entries in the Plex DB
"""
with PlexDB() as plexdb:
plexdb.wipe_playlists()
def _m3u_iterator(text): def _m3u_iterator(text):
""" """
Yields e.g. plugin://plugin.video.plexkodiconnect.movies/?plex_id=xxx Yields e.g. plugin://plugin.video.plexkodiconnect.movies/?plex_id=xxx

15
resources/lib/playlists/kodi_pl.py
View file

@ -96,6 +96,21 @@ def delete(playlist):
db.update_playlist(playlist, delete=True) db.update_playlist(playlist, delete=True)
def delete_kodi_playlists(playlist_paths):
"""
Deletes all the the playlist files passed in; WILL IGNORE THIS CHANGE ON
THE PLEX SIDE!
"""
for path in playlist_paths:
try:
path_ops.remove(path)
# Ensure we're not deleting the playlists on the Plex side later
IGNORE_KODI_PLAYLIST_CHANGE.append(path)
LOG.info('Removed playlist %s', path)
except (OSError, IOError):
LOG.warn('Could not remove playlist %s', path)
def _write_playlist_to_file(playlist, xml): def _write_playlist_to_file(playlist, xml):
""" """
Feed with playlist Playlist. Will write the playlist to a m3u file Feed with playlist Playlist. Will write the playlist to a m3u file

13
resources/lib/playqueue.py
View file

@ -120,7 +120,7 @@ class PlayqueueMonitor(backgroundthread.KillableThread):
# Ignore new media added by other addons # Ignore new media added by other addons
continue continue
for j, old_item in enumerate(old): for j, old_item in enumerate(old):
if self.isCanceled(): if self.should_suspend() or self.should_cancel():
# Chances are that we got an empty Kodi playlist due to # Chances are that we got an empty Kodi playlist due to
# Kodi exit # Kodi exit
return return
@ -189,7 +189,7 @@ class PlayqueueMonitor(backgroundthread.KillableThread):
for j in range(i, len(index)): for j in range(i, len(index)):
index[j] += 1 index[j] += 1
for i in reversed(index): for i in reversed(index):
if self.isCanceled(): if self.should_suspend() or self.should_cancel():
# Chances are that we got an empty Kodi playlist due to # Chances are that we got an empty Kodi playlist due to
# Kodi exit # Kodi exit
return return
@ -212,9 +212,10 @@ class PlayqueueMonitor(backgroundthread.KillableThread):
LOG.info("----===## PlayqueueMonitor stopped ##===----") LOG.info("----===## PlayqueueMonitor stopped ##===----")
def _run(self): def _run(self):
while not self.isCanceled(): while not self.should_cancel():
if self.wait_while_suspended(): if self.should_suspend():
return if self.wait_while_suspended():
return
with app.APP.lock_playqueues: with app.APP.lock_playqueues:
for playqueue in PLAYQUEUES: for playqueue in PLAYQUEUES:
kodi_pl = js.playlist_get_items(playqueue.playlistid) kodi_pl = js.playlist_get_items(playqueue.playlistid)
@ -228,4 +229,4 @@ class PlayqueueMonitor(backgroundthread.KillableThread):
# compare old and new playqueue # compare old and new playqueue
self._compare_playqueues(playqueue, kodi_pl) self._compare_playqueues(playqueue, kodi_pl)
playqueue.old_kodi_pl = list(kodi_pl) playqueue.old_kodi_pl = list(kodi_pl)
app.APP.monitor.waitForAbort(0.2) self.sleep(0.2)

11
resources/lib/plex_companion.py
View file

@ -293,7 +293,7 @@ class PlexCompanion(backgroundthread.KillableThread):
subscription_manager, subscription_manager,
('', v.COMPANION_PORT), ('', v.COMPANION_PORT),
listener.MyHandler) listener.MyHandler)
httpd.timeout = 0.95 httpd.timeout = 10.0
break break
except Exception: except Exception:
LOG.error("Unable to start PlexCompanion. Traceback:") LOG.error("Unable to start PlexCompanion. Traceback:")
@ -312,12 +312,13 @@ class PlexCompanion(backgroundthread.KillableThread):
if httpd: if httpd:
thread = Thread(target=httpd.handle_request) thread = Thread(target=httpd.handle_request)
while not self.isCanceled(): while not self.should_cancel():
# If we are not authorized, sleep # If we are not authorized, sleep
# Otherwise, we trigger a download which leads to a # Otherwise, we trigger a download which leads to a
# re-authorizations # re-authorizations
if self.wait_while_suspended(): if self.should_suspend():
break if self.wait_while_suspended():
break
try: try:
message_count += 1 message_count += 1
if httpd: if httpd:
@ -356,6 +357,6 @@ class PlexCompanion(backgroundthread.KillableThread):
app.APP.companion_queue.task_done() app.APP.companion_queue.task_done()
# Don't sleep # Don't sleep
continue continue
app.APP.monitor.waitForAbort(0.05) self.sleep(0.05)
subscription_manager.signal_stop() subscription_manager.signal_stop()
client.stop_all() client.stop_all()

7
resources/lib/plex_db/common.py
View file

@ -20,18 +20,19 @@ SUPPORTED_KODI_TYPES = (
class PlexDBBase(object): class PlexDBBase(object):
""" """
Plex database methods used for all types of items Plex database methods used for all types of items.
""" """
def __init__(self, plexconn=None, lock=True): def __init__(self, plexconn=None, lock=True, copy=False):
# Allows us to use this class with a cursor instead of context mgr # Allows us to use this class with a cursor instead of context mgr
self.plexconn = plexconn self.plexconn = plexconn
self.cursor = self.plexconn.cursor() if self.plexconn else None self.cursor = self.plexconn.cursor() if self.plexconn else None
self.lock = lock self.lock = lock
self.copy = copy
def __enter__(self): def __enter__(self):
if self.lock: if self.lock:
PLEXDB_LOCK.acquire() PLEXDB_LOCK.acquire()
self.plexconn = db.connect('plex') self.plexconn = db.connect('plex-copy' if self.copy else 'plex')
self.cursor = self.plexconn.cursor() self.cursor = self.plexconn.cursor()
return self return self

13
resources/lib/plex_db/playlists.py
View file

@ -81,3 +81,16 @@ class Playlists(object):
playlist.kodi_type = answ[4] playlist.kodi_type = answ[4]
playlist.kodi_hash = answ[5] playlist.kodi_hash = answ[5]
return playlist return playlist
def all_kodi_paths(self):
"""
Returns a generator for all kodi_paths of all synched playlists
"""
self.cursor.execute('SELECT kodi_path FROM playlists')
return (x[0] for x in self.cursor)
def wipe_playlists(self):
"""
Deletes all entries in the playlists table
"""
self.cursor.execute('DELETE FROM playlists')

1
resources/lib/plexbmchelper/listener.py
View file

@ -37,6 +37,7 @@ RESOURCES_XML = ('%s<MediaContainer>\n'
v.PLATFORM, v.PLATFORM,
v.PLATFORM_VERSION) v.PLATFORM_VERSION)
class MyHandler(BaseHTTPRequestHandler): class MyHandler(BaseHTTPRequestHandler):
""" """
BaseHTTPRequestHandler implementation of Plex Companion listener BaseHTTPRequestHandler implementation of Plex Companion listener

5
resources/lib/service_entry.py
View file

@ -101,7 +101,7 @@ class Service(object):
self._init_done = True self._init_done = True
@staticmethod @staticmethod
def isCanceled(): def should_cancel():
return xbmc.abortRequested or app.APP.stop_pkc return xbmc.abortRequested or app.APP.stop_pkc
def on_connection_check(self, result): def on_connection_check(self, result):
@ -437,7 +437,7 @@ class Service(object):
self.playqueue = playqueue.PlayqueueMonitor() self.playqueue = playqueue.PlayqueueMonitor()
# Main PKC program loop # Main PKC program loop
while not self.isCanceled(): while not self.should_cancel():
# Check for PKC commands from other Python instances # Check for PKC commands from other Python instances
plex_command = utils.window('plexkodiconnect.command') plex_command = utils.window('plexkodiconnect.command')
@ -544,6 +544,7 @@ class Service(object):
# Tell all threads to terminate (e.g. several lib sync threads) # Tell all threads to terminate (e.g. several lib sync threads)
LOG.debug('Aborting all threads') LOG.debug('Aborting all threads')
app.APP.stop_pkc = True app.APP.stop_pkc = True
backgroundthread.BGThreader.shutdown(block=False)
# Load/Reset PKC entirely - important for user/Kodi profile switch # Load/Reset PKC entirely - important for user/Kodi profile switch
# Clear video nodes properties # Clear video nodes properties
library_sync.clear_window_vars() library_sync.clear_window_vars()

21
resources/lib/sync.py
View file

@ -38,7 +38,9 @@ class Sync(backgroundthread.KillableThread):
self.start_library_sync(show_dialog=True, self.start_library_sync(show_dialog=True,
repair=app.SYNC.run_lib_scan == 'repair', repair=app.SYNC.run_lib_scan == 'repair',
block=True) block=True)
if not self.sync_successful and not self.isSuspended() and not self.isCanceled(): if (not self.sync_successful and
not self.should_suspend() and
not self.should_cancel()):
# ERROR in library sync # ERROR in library sync
LOG.warn('Triggered full/repair sync has not been successful') LOG.warn('Triggered full/repair sync has not been successful')
elif app.SYNC.run_lib_scan == 'fanart': elif app.SYNC.run_lib_scan == 'fanart':
@ -112,7 +114,7 @@ class Sync(backgroundthread.KillableThread):
LOG.info('Not synching Plex artwork - not caching') LOG.info('Not synching Plex artwork - not caching')
return return
if self.image_cache_thread and self.image_cache_thread.is_alive(): if self.image_cache_thread and self.image_cache_thread.is_alive():
self.image_cache_thread.abort() self.image_cache_thread.cancel()
self.image_cache_thread.join() self.image_cache_thread.join()
self.image_cache_thread = artwork.ImageCachingThread() self.image_cache_thread = artwork.ImageCachingThread()
self.image_cache_thread.start() self.image_cache_thread.start()
@ -163,10 +165,11 @@ class Sync(backgroundthread.KillableThread):
utils.init_dbs() utils.init_dbs()
while not self.isCanceled(): while not self.should_cancel():
# In the event the server goes offline # In the event the server goes offline
if self.wait_while_suspended(): if self.should_suspend():
return if self.wait_while_suspended():
return
if not install_sync_done: if not install_sync_done:
# Very FIRST sync ever upon installation or reset of Kodi DB # Very FIRST sync ever upon installation or reset of Kodi DB
LOG.info('Initial start-up full sync starting') LOG.info('Initial start-up full sync starting')
@ -188,7 +191,7 @@ class Sync(backgroundthread.KillableThread):
self.start_image_cache_thread() self.start_image_cache_thread()
else: else:
LOG.error('Initial start-up full sync unsuccessful') LOG.error('Initial start-up full sync unsuccessful')
app.APP.monitor.waitForAbort(1) self.sleep(1)
xbmc.executebuiltin('InhibitIdleShutdown(false)') xbmc.executebuiltin('InhibitIdleShutdown(false)')
elif not initial_sync_done: elif not initial_sync_done:
@ -205,7 +208,7 @@ class Sync(backgroundthread.KillableThread):
self.start_image_cache_thread() self.start_image_cache_thread()
else: else:
LOG.info('Startup sync has not yet been successful') LOG.info('Startup sync has not yet been successful')
app.APP.monitor.waitForAbort(1) self.sleep(1)
# Currently no db scan, so we could start a new scan # Currently no db scan, so we could start a new scan
else: else:
@ -240,9 +243,9 @@ class Sync(backgroundthread.KillableThread):
library_sync.store_websocket_message(message) library_sync.store_websocket_message(message)
queue.task_done() queue.task_done()
# Sleep just a bit # Sleep just a bit
app.APP.monitor.waitForAbort(0.01) self.sleep(0.01)
continue continue
app.APP.monitor.waitForAbort(0.1) self.sleep(0.1)
# Shut down playlist monitoring # Shut down playlist monitoring
if playlist_monitor: if playlist_monitor:
playlist_monitor.stop() playlist_monitor.stop()

36
resources/lib/utils.py
View file

@ -249,6 +249,15 @@ def ERROR(txt='', hide_tb=False, notify=False, cancel_sync=False):
return short return short
def rreplace(s, old, new, occurrence=-1):
"""
Replaces the string old [str, unicode] with new from the RIGHT given a
string s.
"""
li = s.rsplit(old, occurrence)
return new.join(li)
class AttributeDict(dict): class AttributeDict(dict):
""" """
Turns an etree xml response's xml.attrib into an object with attributes Turns an etree xml response's xml.attrib into an object with attributes
@ -525,29 +534,6 @@ def delete_temporary_subtitles():
root, file, err) root, file, err)
def wipe_synched_playlists():
"""
Deletes all synched playlist files on the Kodi side; resets the Plex table
listing all synched Plex playlists
"""
from . import plex_db
try:
with plex_db.PlexDB() as plexdb:
plexdb.cursor.execute('SELECT kodi_path FROM playlists')
playlist_paths = [x[0] for x in plexdb.cursor]
except OperationalError:
# Plex DB completely empty yet
playlist_paths = []
for path in playlist_paths:
try:
path_ops.remove(path)
LOG.info('Removed playlist %s', path)
except (OSError, IOError):
LOG.warn('Could not remove playlist %s', path)
# Now wipe our database
plex_db.wipe(table='playlists')
def wipe_database(reboot=True): def wipe_database(reboot=True):
""" """
Deletes all Plex playlists as well as video nodes, then clears Kodi as well Deletes all Plex playlists as well as video nodes, then clears Kodi as well
@ -557,10 +543,10 @@ def wipe_database(reboot=True):
LOG.warn('Start wiping') LOG.warn('Start wiping')
from .library_sync.sections import delete_files from .library_sync.sections import delete_files
from . import kodi_db, plex_db from . import kodi_db, plex_db
from .playlists import remove_synced_playlists
# Clean up the playlists and video nodes # Clean up the playlists and video nodes
delete_files() delete_files()
# Wipe all synched playlists remove_synced_playlists()
wipe_synched_playlists()
try: try:
with plex_db.PlexDB() as plexdb: with plex_db.PlexDB() as plexdb:
if plexdb.songs_have_been_synced(): if plexdb.songs_have_been_synced():

10
resources/lib/variables.py
View file

@ -69,7 +69,14 @@ elif xbmc.getCondVisibility('system.platform.android'):
else: else:
DEVICE = "Unknown" DEVICE = "Unknown"
MODEL = platform.release() or 'Unknown' try:
MODEL = platform.release() or 'Unknown'
except IOError:
# E.g. iOS
# It seems that Kodi doesn't allow python to spawn subprocesses in order to
# determine the system name
# See https://github.com/psf/requests/issues/4434
MODEL = 'Unknown'
DEVICENAME = try_decode(_ADDON.getSetting('deviceName')) DEVICENAME = try_decode(_ADDON.getSetting('deviceName'))
if not DEVICENAME: if not DEVICENAME:
@ -127,6 +134,7 @@ DB_MUSIC_PATH = None
DB_TEXTURE_VERSION = None DB_TEXTURE_VERSION = None
DB_TEXTURE_PATH = None DB_TEXTURE_PATH = None
DB_PLEX_PATH = try_decode(xbmc.translatePath("special://database/plex.db")) DB_PLEX_PATH = try_decode(xbmc.translatePath("special://database/plex.db"))
DB_PLEX_COPY_PATH = try_decode(xbmc.translatePath("special://database/plex-copy.db"))
EXTERNAL_SUBTITLE_TEMP_PATH = try_decode(xbmc.translatePath( EXTERNAL_SUBTITLE_TEMP_PATH = try_decode(xbmc.translatePath(
"special://profile/addon_data/%s/temp/" % ADDON_ID)) "special://profile/addon_data/%s/temp/" % ADDON_ID))

28
resources/lib/websocket_client.py
View file

@ -19,7 +19,7 @@ class WebSocket(backgroundthread.KillableThread):
def __init__(self): def __init__(self):
self.ws = None self.ws = None
self.redirect_uri = None self.redirect_uri = None
self.sleeptime = 0 self.sleeptime = 0.0
super(WebSocket, self).__init__() super(WebSocket, self).__init__()
def process(self, opcode, message): def process(self, opcode, message):
@ -46,15 +46,15 @@ class WebSocket(backgroundthread.KillableThread):
def getUri(self): def getUri(self):
raise NotImplementedError raise NotImplementedError
def __sleep(self): def _sleep_cycle(self):
""" """
Sleeps for 2^self.sleeptime where sleeping period will be doubled with Sleeps for 2^self.sleeptime where sleeping period will be doubled with
each unsuccessful connection attempt. each unsuccessful connection attempt.
Will sleep at most 64 seconds Will sleep at most 64 seconds
""" """
app.APP.monitor.waitForAbort(2**self.sleeptime) self.sleep(2 ** self.sleeptime)
if self.sleeptime < 6: if self.sleeptime < 6:
self.sleeptime += 1 self.sleeptime += 1.0
def run(self): def run(self):
LOG.info("----===## Starting %s ##===----", self.__class__.__name__) LOG.info("----===## Starting %s ##===----", self.__class__.__name__)
@ -69,9 +69,9 @@ class WebSocket(backgroundthread.KillableThread):
LOG.info("##===---- %s Stopped ----===##", self.__class__.__name__) LOG.info("##===---- %s Stopped ----===##", self.__class__.__name__)
def _run(self): def _run(self):
while not self.isCanceled(): while not self.should_cancel():
# In the event the server goes offline # In the event the server goes offline
if self.isSuspended(): if self.should_suspend():
# Set in service.py # Set in service.py
if self.ws is not None: if self.ws is not None:
self.ws.close() self.ws.close()
@ -99,11 +99,11 @@ class WebSocket(backgroundthread.KillableThread):
# Server is probably offline # Server is probably offline
LOG.debug("%s: IOError connecting", self.__class__.__name__) LOG.debug("%s: IOError connecting", self.__class__.__name__)
self.ws = None self.ws = None
self.__sleep() self._sleep_cycle()
except websocket.WebSocketTimeoutException: except websocket.WebSocketTimeoutException:
LOG.debug("%s: WebSocketTimeoutException", self.__class__.__name__) LOG.debug("%s: WebSocketTimeoutException", self.__class__.__name__)
self.ws = None self.ws = None
self.__sleep() self._sleep_cycle()
except websocket.WebsocketRedirect as e: except websocket.WebsocketRedirect as e:
LOG.debug('301 redirect detected: %s', e) LOG.debug('301 redirect detected: %s', e)
self.redirect_uri = e.headers.get('location', self.redirect_uri = e.headers.get('location',
@ -111,11 +111,11 @@ class WebSocket(backgroundthread.KillableThread):
if self.redirect_uri: if self.redirect_uri:
self.redirect_uri = self.redirect_uri.decode('utf-8') self.redirect_uri = self.redirect_uri.decode('utf-8')
self.ws = None self.ws = None
self.__sleep() self._sleep_cycle()
except websocket.WebSocketException as e: except websocket.WebSocketException as e:
LOG.debug('%s: WebSocketException: %s', self.__class__.__name__, e) LOG.debug('%s: WebSocketException: %s', self.__class__.__name__, e)
self.ws = None self.ws = None
self.__sleep() self._sleep_cycle()
except Exception as e: except Exception as e:
LOG.error('%s: Unknown exception encountered when ' LOG.error('%s: Unknown exception encountered when '
'connecting: %s', self.__class__.__name__, e) 'connecting: %s', self.__class__.__name__, e)
@ -123,9 +123,9 @@ class WebSocket(backgroundthread.KillableThread):
LOG.error("%s: Traceback:\n%s", LOG.error("%s: Traceback:\n%s",
self.__class__.__name__, traceback.format_exc()) self.__class__.__name__, traceback.format_exc())
self.ws = None self.ws = None
self.__sleep() self._sleep_cycle()
else: else:
self.sleeptime = 0 self.sleeptime = 0.0
except Exception as e: except Exception as e:
LOG.error("%s: Unknown exception encountered: %s", LOG.error("%s: Unknown exception encountered: %s",
self.__class__.__name__, e) self.__class__.__name__, e)
@ -141,7 +141,7 @@ class PMS_Websocket(WebSocket):
""" """
Websocket connection with the PMS for Plex Companion Websocket connection with the PMS for Plex Companion
""" """
def isSuspended(self): def should_suspend(self):
""" """
Returns True if the thread is suspended Returns True if the thread is suspended
""" """
@ -206,7 +206,7 @@ class Alexa_Websocket(WebSocket):
""" """
Websocket connection to talk to Amazon Alexa. Websocket connection to talk to Amazon Alexa.
""" """
def isSuspended(self): def should_suspend(self):
""" """
Overwrite method since we need to check for plex token Overwrite method since we need to check for plex token
""" """

8
resources/lib/windows/userselect.py
View file

@ -24,7 +24,7 @@ class UserThumbTask(backgroundthread.Task):
def run(self): def run(self):
for user in self.users: for user in self.users:
if self.isCanceled(): if self.should_cancel():
return return
thumb, back = user.thumb, '' thumb, back = user.thumb, ''
self.callback(user, thumb, back) self.callback(user, thumb, back)
@ -169,13 +169,13 @@ class UserSelectWindow(kodigui.BaseWindow):
utils.settings('plexToken'), utils.settings('plexToken'),
utils.settings('plex_machineIdentifier')) utils.settings('plex_machineIdentifier'))
if self.user.authToken is None: if self.user.authToken is None:
self.user = None
item.setProperty('pin', item.dataSource.title) item.setProperty('pin', item.dataSource.title)
item.setProperty('editing.pin', '') item.setProperty('editing.pin', '')
# 'Error': 'Login failed with plex.tv for user' # 'Error': 'Login failed with plex.tv for user'
utils.messageDialog(utils.lang(30135), utils.messageDialog(utils.lang(30135),
'%s %s' % (utils.lang(39229), '{}{}'.format(utils.lang(39229),
self.user.username)) self.user.username))
self.user = None
return return
self.doClose() self.doClose()