# (c) Copyright 2009-2012, 2015. CodeWeavers, Inc.

"""This is the Application Installation Engine, AIE for short.

Based on the InstallTask data, it is responsible for determining which tasks
will be needed to perform the installation, and what their dependencies are.

Note that the installation profiles form a general directed graph. So it is
necessary to have in mind this graph in order to understand the what the
engine does.

Here are the tasks it may create:
 * One task is created to install a each application. The dependencies between
   these tasks reflect the dependencies between the install profiles. Obviously
   cycles cannot be allowed so they must be detected and reported.

 * If the installer needs to be downloaded then a separate task is created to
   handle that.

 * If the source media needs to be specified for an application, then a set
   of tasks is created to handle its insertion and release.

 * If a downloadable installer may also be found on CD, then the relevant
   download task is 'linked' to the compatible media sources, where the notion
   of 'compatibility' stems from the necessity to avoid introducing loops in
   the task graph.

 * Finally, additional tasks are created to handle the bottle creation, the
   pre-installation tasks and the post-installation tasks.


Here's a more detailed description of how the engine transforms the install
profile graph into an installation tasks graph:

1. Get the aggregated installer profiles of all the applications to
   install from the parent InstallTask.

2. If the main application comes on CD but there are applications to
   install first that come on a CD too (potentially), then the first
   thing we will do is ask the user to eject this CD before we've had
   time to scan it to see if there might be something useful on it.
   We may still get an opportunity to scan it when it gets inserted
   later, but that may be too late for some applications and will
   in any case delay the start of the downloads. So create a dummy
   AIECore we can attach this 'initial media' to so we can handle it
   normally.

   If installtask says we're installing from a directory, then:
   - Create an AIECore object for a fake 'DummyApp' application.
   - Call its done() method so it is not actually run and does not
     prevent other tasks from running.
   - Associate a media triplet to the AIECore object.
   - Call the AIEInsertMedia object's done() method so it is not
     actually run (and thus does not ask the user to insert the
     already present CD), but does not prevent other tasks from
     running.

3. Create an initial set of tasks for each application to install.
   This includes the tasks directly connected to these applications
   like tasks to download them, to insert the media they come on,
   etc.

   We do that through a recursive depth-first traversal starting from
   the main application profile and following the dependency links,
   ignoring installed applications. This really needs a recursion
   because we need to gather data both as we go down and up the
   recursion path.
   As we recurse:
   - If an AIECore object exists for the current application already (look
     it up in the global aiecores map), then:
     - Make the parent depend on it.
     - If parent media is set, then add it to the parent-media set of all
       AIECore objects in the children-scan set (note that if this object
       has a media, then its children-scan set is empty).
     - Return the cached result tuple.

   - Otherwise, create an AIECore object and add it to the global aiecores map.
   - Associate the AIECore object with the corresponding installer profile.
   - Make the AIECore object depend on the PreInstall task.
   - Make the parent depend on the new AIECore object.

   - If the application installer is already available skip to the next
     group.
   - Else if the application can be downloaded, create an AIEDownload object
     and make the AIECore object depend on it. If the LocalInstallerFileGlobs
     field is set, add the AIECore object to the global scan list.
   - Else create an (AIEInsertMedia, AIEScanMedia, AIEReleaseMedia) triplet
     with meaningful dependencies between them and the AIECore object and
     set parent-media to the current object.

   Then recursively process the dependencies. For each of the dependencies
   we get back a tuple of (children-scan set, children-media set,
   descendents-media set) containing AIECore objects. With them:
   - Aggregate the children-scan sets into a single set.
   - Aggregate the children-media sets into a single set.
   - Aggregate the descendents-media sets into a single set.

   On the recursion return path:
   - If the current object has an AIEInsertMedia object, then:
     - If DummyApp exists, the current object is the not main profile
       and the children-media set is empty, then add the DummyApp AIECore
       object to the chidren-media and descendents-media sets.
     - Make this object's AIEInsertMedia depend on the AIEReleaseMedia
       objects of all the AIECores in the children-media set.
     - Reset the children-media set so it contains only the current object.
     - Make the children-scan set empty.
     - Add the current object to the descendents-media set.
   - If it does not have an AIEInsertMedia object but its descendents-media
     set is non-empty, then add it to the parent-media set of all the objects
     in its children-scan set.
   - If the LocalInstallerFileGlobs field is set for the current profile,
     then add the AIECore object to the children-scan set.
   - Store the (children-scan set, children-media set, descendents-media
     set) triplet in the result mapping based on the AIECore object.
     This is the result triplet.
   - Finally, return the result triplet to the caller.

4. The objects in the global scan list may be able to find their
   installer on a CD instead of having to download them. So link
   them to the compatible AIEScanMedia objects.

   For each object in the global scan list:
   - First compute the set of compatible media:
     - Initialize the compatible media set with those objects of the
       AIECore's parent-media set that have a media.
     - If the children-media set is non-empty for the download's
       AIECore object, add them to the compatible media set.
     - Otherwise add the descendents-media set of each of the
       objects in the AIECore object's parent-media set.
   - Then, make the corresponding AIEDownload object depend on the
     AIEScanMedia objects of the objects in the compatible media set.

5. If the AIEScanMedia object of the DummyApp we created in step 3
   has nothing to do then mark it and the AIEReleaseMedia tasks as
   done.

6. Again make sure we will have a chance to scan the CD of the main
   profile before starting the downloads.

   So if the main profile has a media triplet:
   - Detach the AIEScanMedia object from the corresponding
     AIEInsertMedia object.
   - Make the DummyApp's AIEReleaseMedia object depend on this
     AIEScanMedia object. This is so we don't try to insert another
     media before this scan is done.
   - If the main AIECore's AIEInsertMedia object has no dependencies
     (i.e. there is only one CD involved), then arrange for it to
     not trigger asking the user to insert the CD.

7. Compute the AIECore object niceness based on their depth in
   the dependency graph.

8. Make the PostInstall task depend on the main profile AIECore
   object.
"""

import os.path
import os
import shutil
import time

import cxproduct
import cxlog
import cxutils

import appdetector
import bottlequery
import c4profiles
import cxaiecore
import cxaiemedia
import cxaiemisc
import cxaiebase
import cxobjc
import distversion



#####
#
# Task creator and scheduler
#
#####

class Engine(cxaiebase.AIEScheduler):

    #####
    #
    # Installation Logging
    #
    #####

    def start_logging(self, logfile, channels):
        """Initializes the global environment block and sets up logging.

        Setting up logging involves starting cxlogfilter and directing the
        traces of the Wine processes to it. cxlogfilter is responsible for
        ensuring that we don't produce logs that are so huge that they fill the
        disk.

        This function never fails. Logging is there to help diagnosis in case
        things go wrong, not to cause things to go wrong.
        """
        self.state['environ'] = env = os.environ.copy()
        if logfile or 'CX_LOG' not in env:
            if logfile is None:
                # The bottle may not exist yet, so put the temporary log file in the
                # desktopdata directory. This way it won't be too much of a bother if
                # we are unable to move it to the bottle later.
                logdir = os.path.join(cxproduct.get_user_dir(), "desktopdata")
                logfile = os.path.join(logdir, "%s.%d.log" % (self.installtask.profile.appid, os.getpid()))
                self.state['tmplogfile'] = True

            if not cxutils.mkdirs(os.path.dirname(logfile)):
                # We'll just install without logging
                return

            if channels:
                f = None

                try:
                    f = open(logfile, "w")
                except IOError, ioe:
                    cxlog.log("unable to open log file: " + ioe.strerror)
                else:
                    f.write(self.installtask.get_summary_string())
                    f.write('\n')
                    f.close()

                env['CX_LOG'] = self.state['logfile'] = logfile
            else:
                logfifo = logfile + ".fifo"
                cmd = [os.path.join(cxutils.CX_ROOT, "bin", "cxlogfilter"),
                       "--fifo", logfifo, "--log", logfile]
                retcode, _out, _err = cxutils.run(cmd, stdout=cxutils.NULL, stderr=cxutils.NULL)
                if retcode:
                    cxlog.log("unable to start cxlogfilter (%d): %s" % (retcode, _err))
                    return

                env['CX_LOG'] = logfifo

                self.state['logfile'] = logfile
                self.state['logfifo'] = logfifo
                try:
                    f = self.state['logfifo_file'] = open(logfifo, "w")
                except IOError, ioe:
                    cxlog.log("unable to open the fifo: " + ioe.strerror)
                else:
                    f.write(self.installtask.get_summary_string())
                    f.write('\n')
                    f.flush()

        if channels is not None:
            env['CX_DEBUGMSG'] = channels
        elif 'CX_DEBUGMSG' not in env:
            env['CX_DEBUGMSG'] = "+cxreboot,-cxassoc,-cxmenu"


    def stop_logging(self):
        """Stops cxlogfilter and moves the log to its final resting place in
        the bottle.
        """
        logfifo = self.state.get('logfifo')
        if not logfifo:
            return

        logfifo_file = self.state.get('logfifo_file')
        if logfifo_file:
            # Tell cxlogfilter to exit now, even if there are still
            # running processes
            logfifo_file.write(":cxlogfilter:exit\n")
            logfifo_file.close()
            del self.state['logfifo_file']

        # Wait for cxlogfilter to exit (which is indicated by the fifo
        # going away) so the log file is not empty when we show the log to
        # the user. But don't wait too long in case cxlogfilter failed
        # somehow.
        for _dummy in xrange(5):
            if not os.path.exists(logfifo):
                break
            time.sleep(0.2)
        try:
            # Just in case cxlogfilter failed to delete it
            os.unlink(logfifo)
        except OSError:
            # Most likely the fifo was gone already
            pass
        del self.state['logfifo']

        if 'tmplogfile' in self.state:
            # Move the log file to its final resting place
            logfile = self.state['logfile']
            bottlename = self.installtask.bottlename
            # If the bottle is messed up, expand_win_string can throw an exception
            try:
                dstdir = self.expand_win_string("%temp%")
            except IOError:
                dstdir = ""
            if dstdir == "":
                # The bottle creation probably failed or the bottle is messed up
                return

            dstdir = bottlequery.get_native_path(bottlename, dstdir)
            # FIXME: Build a unique filename for the log file
            dst = os.path.join(dstdir, cxutils.sanitize_bottlename(cxlog.to_str(self.installtask.profile.name)) + ".log").decode('utf8')
            try:
                shutil.move(logfile, dst)
                self.state['logfile'] = dst
            except IOError, ioe:
                cxlog.log("unable to move %s to %s: %s" % (cxlog.debug_str(logfile), cxlog.debug_str(dst), ioe.strerror))


    def _getlogfile(self):
        """Returns the path to the installation log if any, and None
        otherwise.
        """
        if 'logfifo' in self.state:
            raise RuntimeError('AIEngine._getlogfile() called before AIEngine.stop_logging()')
        return self.state.get('logfile')

    logfile = property(_getlogfile)


    #####
    #
    # Initialization
    #
    #####

    def _add_profile(self, appid, parent, parent_media):
        cxlog.log("Processing %s" % cxlog.to_str(appid))
        if appid in self._aiecores:
            aiecore = self._aiecores[appid]
            result = self._results[appid]
            if parent:
                parent.add_dependency(aiecore)
            if parent_media:
                children_scan = result[0]
                for scan in children_scan:
                    scan.parent_media.add(parent_media)
            # We've already processed everything down from there
            # so just return the cached result.
            return result

        # Create an AIECore object and insert it into the graph of tasks
        installer = self._installers[appid]
        app_profile = installer.parent.app_profile
        aiecore = cxaiecore.AIECore(self, installer)
        self._aiecores[appid] = aiecore
        aiecore.add_dependency(self._bottle_init)
        if parent:
            parent.add_dependency(aiecore)
        if parent_media:
            aiecore.parent_media.add(parent_media)

        # Some set of descendents we need to keep track of
        children_scan = set()
        children_media = set()
        descendents_media = set()

        # Figure out how to get the installer for this application
        if self.installtask.installWithSteam and app_profile.steamid:
            # No installer needed!
            pass
        elif app_profile.download_urls or \
               cxaiemedia.get_builtin_installer(installer.parent.appid):
            aiecore.create_download_task()
            if aiecore.download.needs_download and installer.local_installer_file_globs:
                self._scan.append(aiecore)
                children_scan.add(aiecore)
        elif app_profile.steamid and \
            u"com.codeweavers.c4.206" in installer.pre_dependencies:
            # Dependency to be installed via steam id
            aiecore.state['install_source'] = "steam://install/%s" % app_profile.steamid
            aiecore.state['installer_file'] = aiecore.state['install_source']
        elif 'virtual' not in app_profile.flags:
            aiecore.create_media_tasks()
            parent_media = aiecore

        # Recursively process the dependencies and aggregate the results.
        # Note that InstallTask has already purged dependency loops so we don't
        # have to worry about them here.
        for depid in installer.pre_dependencies:
            dep_children_scan, dep_children_media, dep_descendents_media = self._add_profile(depid, aiecore, parent_media)
            if not aiecore.insert_media:
                children_scan.update(dep_children_scan)
            children_media.update(dep_children_media)
            descendents_media.update(dep_descendents_media)

        if aiecore.insert_media:
            if self._dummy_app and parent and not descendents_media:
                children_media.add(self._dummy_app)
                descendents_media.add(self._dummy_app)
            # Make sure we don't ask the user to insert this media before the
            # applications it depends on are done with their own media.
            for media in children_media:
                aiecore.insert_media.add_dependency(media.release_media)
            children_media = set((aiecore,))
            descendents_media.add(aiecore)
        elif descendents_media:
            for scan in children_scan:
                scan.parent_media.add(aiecore)

        result = (children_scan, children_media, descendents_media)
        self._results[appid] = result

        # Queue post-dependencies for processing
        for depid in installer.post_dependencies:
            # We don't have to worry about the relationship between this profile
            # and its post-dependency, as InstallTask guarantees the
            # post-dependency already depends on this profile.
            if depid in self._installers: # In case an override removed it
                self._postdeps.add(depid)

        return result

    def _set_depth(self, aiecore, depth):
        # Note that the general task graph _may_ contain dependency loops that
        # will be resolved at run time (i.e. we cannot make the assumption that
        # it does not which would necessitate some precautions). However we
        # know that by construction this cannot happen to the specific subset
        # defined by the AIECore tasks.
        if depth < aiecore.depth:
            aiecore.depth = depth
        if aiecore not in self._parents_left:
            count = 0
            for parent in aiecore.parents:
                if isinstance(parent, cxaiecore.AIECore):
                    count += 1
            self._parents_left[aiecore] = count
        self._parents_left[aiecore] -= 1
        if self._parents_left[aiecore] == 0:
            depth = aiecore.depth - 1
            for child in aiecore.dependencies:
                if isinstance(child, cxaiecore.AIECore):
                    self._set_depth(child, depth)

            # Take this opportunity to let the AIECore finish its
            # initialization
            aiecore.prepare()


    def __init__(self, installtask, log_filename=None, log_channels=None):
        cxaiebase.AIEScheduler.__init__(self)
        self.installtask = installtask
        # Stores the state information used at run time
        self.state = {}

        self._has_done_usage_logging = False
        main_appid = installtask.profile.appid
        profiles = installtask.profiles

        # FIXME: The data from get_installed_applications() should be cached
        # but this is really not the place to do it. If only we were allowed
        # to have bottle objects!
        if installtask.GetCreateNewBottle():
            self.installed = {}
        else:
            self.installed = appdetector.fast_get_installed_applications(installtask.bottlename, profiles)
        try:
            # Don't consider the application we're trying to install as
            # installed
            # FIXME: Once installed is properly cached we'll have to do this
            # some other way (maybe copy installed).
            del self.installed[main_appid]
        except KeyError:
            pass

        # 1. Get the aggregated installer profiles of all the applications to
        #    install from the parent InstallTask.
        self._installers = self.installtask.target_bottle.installers

        # 2. If the main application comes on CD but there are applications to
        #    install first that come on a CD too (potentially), then the first
        #    thing we will do is ask the user to eject this CD before we've had
        #    time to scan it to see if there might be something useful on it.
        #    We may still get an opportunity to scan it when it gets inserted
        #    later, but that may be too late for some applications and will
        #    in any case delay the start of the downloads. So create a dummy
        #    AIECore we can attach this 'initial media' to so we can handle it
        #    normally.
        #
        #    If installtask says we're installing from a directory, then:
        #    - Create an AIECore object for a fake 'DummyApp' application.
        #    - Call its done() method so it is not actually run and does not
        #      prevent other tasks from running.
        #    - Associate a media triplet to the AIECore object.
        #    - Call the AIEInsertMedia object's done() method so it is not
        #      actually run (and thus does not ask the user to insert the
        #      already present CD), but does not prevent other tasks from
        #      running.
        install_source = installtask.GetInstallerSource()
        if install_source and os.path.isdir(install_source):
            self._dummy_app = cxaiecore.AIECore(self, profiles.unknown_installer().copy())
            self._dummy_app.create_media_tasks()
            # Make sure neither dummy_app nor its insert_media task will run
            # pylint: disable=E1101
            self._dummy_app.done()
            self._dummy_app.insert_media.install_source = install_source
            self._dummy_app.insert_media.done()
        else:
            self._dummy_app = None

        # 3. Create an initial set of tasks for each application to install.
        #    This includes the tasks directly connected to these applications
        #    like tasks to download them, to insert the media they come on,
        #    etc.
        #
        #    We do that through a recursive depth-first traversal starting from
        #    the main application profile and following the dependency links,
        #    ignoring installed applications. This really needs a recursion
        #    because we need to gather data both as we go down and up the
        #    recursion path.
        self._bottle_init = cxaiemisc.get_bottle_init_task(self)
        self._aiecores = {}
        self._scan = []
        self._results = {}
        self._postdeps = set()
        self._add_profile(main_appid, None, None)

        while self._postdeps:
            self._add_profile(self._postdeps.pop(), None, None)

        # 4. The objects in the global scan list may be able to find their
        #    installer on a CD instead of having to download them. So link
        #    them to the compatible AIEScanMedia objects.
        #
        #    For each object in the global scan list:
        #    - First compute the set of compatible media:
        #      - Initialize the compatible media set with those objects of the
        #        AIECore's parent-media set that have a media.
        #      - If the children-media set is non-empty for the download's
        #        AIECore object, add them to the compatible media set.
        #      - Otherwise add the descendents-media set of each of the
        #        objects in the AIECore object's parent-media set.
        #    - Then, make the corresponding AIEDownload object depend on the
        #      AIEScanMedia objects of the objects in the compatible media set.
        for aiecore in self._scan:
            _children_scan, children_media, _descendents_media = self._results[aiecore.installer.parent.appid]
            if children_media:
                compatible_media = children_media.copy()
                for parent in aiecore.parent_media:
                    if parent.insert_media:
                        compatible_media.add(parent)
            else:
                compatible_media = set()
                for parent in aiecore.parent_media:
                    _children_scan, _children_media, descendents_media = self._results[parent.installer.parent.appid]
                    compatible_media.update(descendents_media)

            for media in compatible_media:
                aiecore.download.add_dependency(media.scan_media)

        # 5. If the AIEScanMedia object of the DummyApp we created in step 3
        #    has nothing to do then mark it and the AIEReleaseMedia tasks as
        #    done.
        aiecore = self._aiecores[main_appid]
        if self._dummy_app and len(self._dummy_app.scan_media.parents) == 1:
            self._dummy_app.scan_media.done()
            self._dummy_app.release_media.done()
            self._dummy_app = None

        # 6. Again make sure we will have a chance to scan the CD of the main
        #    profile before starting the downloads.
        #
        #    So if the main profile has a media triplet:
        #    - Detach the AIEScanMedia object from the corresponding
        #      AIEInsertMedia object.
        #    - Make the DummyApp's AIEReleaseMedia object depend on this
        #      AIEScanMedia object. This is so we don't try to insert another
        #      media before this scan is done.
        #    - If the main AIECore's AIEInsertMedia object has no dependencies
        #      (i.e. there is only one CD involved), then arrange for it to
        #      not trigger asking the user to insert the CD.
        if aiecore.insert_media:
            aiecore.scan_media.remove_dependency(aiecore.insert_media)
            if self._dummy_app:
                self._dummy_app.release_media.add_dependency(aiecore.scan_media)
            if not aiecore.insert_media.dependencies:
                aiecore.insert_media.install_source = install_source
                aiecore.insert_media.detect = False
        if install_source:
            aiecore.state['install_source'] = install_source

        # 7. Compute the AIECore object niceness based on their depth in
        #    the dependency graph.
        self._parents_left = {aiecore: 1}
        self._set_depth(aiecore, 0)

        # 8. Make the PostInstall task depend on all AIECore objects.
        post_install = cxaiemisc.get_postinstall_task(self)
        for aiecore in self._aiecores.itervalues():
            post_install.add_dependency(aiecore)

        # 9. Initialize logging
        self.start_logging(log_filename, log_channels)

        # On the Mac, prevent cycling the MIDI server, since launchd throttles
        # the rate at which it can be restarted.
        if distversion.IS_MACOSX:
            self.state['environ']['CX_DISABLE_COREAUDIO_MIDI'] = '1'

        # 10. We can clean up temporary variables and do consistency checks
        del self._bottle_init
        del self._scan
        del self._results
        del self._parents_left

        if cxlog.is_on():
            self.check("Engine.__init__()", dump=cxlog.is_on("aie"))
            cxlog.log_("aie", "Sorted task list:")
            for task in self.get_sorted_tasks():
                cxlog.log_("aie", "  " + cxlog.to_str(task))


    # Special initializer for objc. This must always be called explicitly
    # on the Mac.
    def initWithInstallTask_logFile_channels_(self, installtask, log_filename, log_channels):
        self = cxobjc.Proxy.nsobject_init(self)
        if self is not None:
            self.__init__(installtask, log_filename, log_channels)
        return self


    #####
    #
    # Other overrides
    #
    #####

    def all_done(self):
        """Same as AIEScheduler.all_done(), but also takes care of:
           1.) stopping the logging; and
           2.) writing to the usage log if necessary
        """
        all_done = cxaiebase.AIEScheduler.all_done(self)
        if all_done:
            self.stop_logging()
            if not self._has_done_usage_logging:
                for aiecore in self._aiecores.itervalues():
                    aiecore.log_usage()
                self._has_done_usage_logging = True

        return all_done


    #####
    #
    # Helpers
    #
    #####

    def get_win_environ(self):
        """Initializes the 'winenv' state variable and returns it."""
        if 'winenv' not in self.state:
            self.state['winenv'] = bottlequery.get_win_environ(self.installtask.bottlename, c4profiles.ENVIRONMENT_VARIABLES)
        return self.state['winenv']


    def expand_win_string(self, string):
        """Expands the references to the authorized Windows environment
        variables.
        """
        return bottlequery.expand_win_string(self.get_win_environ(), string)