Knapsack Problems - Numerical Optimization

Skip to content
Back to top
View this page
Edit this page

# Knapsack Problems

This module implements a number of solutions to various knapsack problems, otherwise known as linear integer programming problems. Solutions to the following knapsack problems are implemented:

- Solving the subset sum problem for super-increasing sequences.

- General case using Linear Programming

AUTHORS:

- Minh Van Nguyen (2009-04): initial version

- Nathann Cohen (2009-08): Linear Programming version

## Definition of Knapsack problems

You have already had a knapsack problem, so you should know, but in case you do not, a knapsack problem is what happens when you have hundred of items to put into a bag which is too small, and you want to pack the most useful of them.

When you formally write it, here is your problem:

- Your bag can contain a weight of at most \(W\).

- Each item \(i\) has a weight \(w_i\).

- Each item \(i\) has a usefulness \(u_i\).

You then want to maximize the total usefulness of the items you will store into your bag, while keeping sure the weight of the bag will not go over \(W\).

As a linear program, this problem can be represented this way (if you define \(b_i\) as the binary variable indicating whether the item \(i\) is to be included in your bag):

\[\begin{split}\mbox{Maximize: }\sum_i b_i u_i \\ \mbox{Such that: } \sum_i b_i w_i \leq W \\ \forall i, b_i \mbox{ binary variable} \\\end{split}\]

(For more information, see the Wikipedia article Knapsack_problem)

## Examples

If your knapsack problem is composed of three items (weight, value) defined by (1,2), (1.5,1), (0.5,3), and a bag of maximum weight 2, you can easily solve it this way:

Sage

    sage: from sage.numerical.knapsack import knapsack
    sage: knapsack( [(1,2), (1.5,1), (0.5,3)], max=2)
    [5.0, [(1, 2), (0.500000000000000, 3)]]

Python

    >>> from sage.all import *
    >>> from sage.numerical.knapsack import knapsack
    >>> knapsack( [(Integer(1),Integer(2)), (RealNumber('1.5'),Integer(1)), (RealNumber('0.5'),Integer(3))], max=Integer(2))
    [5.0, [(1, 2), (0.500000000000000, 3)]]

## Super-increasing sequences

We can test for whether or not a sequence is super-increasing:

Sage

    sage: from sage.numerical.knapsack import Superincreasing
    sage: L = [1, 2, 5, 21, 69, 189, 376, 919]
    sage: seq = Superincreasing(L)
    sage: seq
    Super-increasing sequence of length 8
    sage: seq.is_superincreasing()
    True
    sage: Superincreasing().is_superincreasing([1,3,5,7])
    False

Python

    >>> from sage.all import *
    >>> from sage.numerical.knapsack import Superincreasing
    >>> L = [Integer(1), Integer(2), Integer(5), Integer(21), Integer(69), Integer(189), Integer(376), Integer(919)]
    >>> seq = Superincreasing(L)
    >>> seq
    Super-increasing sequence of length 8
    >>> seq.is_superincreasing()
    True
    >>> Superincreasing().is_superincreasing([Integer(1),Integer(3),Integer(5),Integer(7)])
    False

Solving the subset sum problem for a super-increasing sequence and target sum:

Sage

    sage: L = [1, 2, 5, 21, 69, 189, 376, 919]
    sage: Superincreasing(L).subset_sum(98)
    [69, 21, 5, 2, 1]

[Python]

    >>> from sage.all import *
    >>> L = [Integer(1), Integer(2), Integer(5), Integer(21), Integer(69), Integer(189), Integer(376), Integer(919)]
    >>> Superincreasing(L).subset_sum(Integer(98))
    [69, 21, 5, 2, 1]

class sage.numerical.knapsack.Superincreasing(seq=None)

  Bases: `SageObject`

  A class for super-increasing sequences.

  Let \(L = (a_1, a_2, a_3, \dots, a_n)\) be a non-empty sequence of nonnegative integers. Then \(L\) is said to be super-increasing if each \(a_i\) is strictly greater than the sum of all previous values. That is, for each \(a_i \in L\) the sequence \(L\) must satisfy the property

  \[a_i > \sum_{k=1}^{i-1} a_k\]

  in order to be called a super-increasing sequence, where \(|L| \geq 2\). If \(L\) has only one element, it is also defined to be a super-increasing sequence.

  If `seq` is `None`, then construct an empty sequence. By definition, this empty sequence is not super-increasing.

  INPUT:

  - `seq` – (default: `None`) a non-empty sequence

  EXAMPLES:

  Sage

      sage: from sage.numerical.knapsack import Superincreasing
      sage: L = [1, 2, 5, 21, 69, 189, 376, 919]
      sage: Superincreasing(L).is_superincreasing()
      True
      sage: Superincreasing().is_superincreasing([1,3,5,7])
      False
      sage: seq = Superincreasing(); seq
      An empty sequence.
      sage: seq = Superincreasing([1, 3, 6]); seq
      Super-increasing sequence of length 3
      sage: seq = Superincreasing([1, 2, 5, 21, 69, 189, 376, 919]); seq
      Super-increasing sequence of length 8

  Python

      >>> from sage.all import *
      >>> from sage.numerical.knapsack import Superincreasing
      >>> L = [Integer(1), Integer(2), Integer(5), Integer(21), Integer(69), Integer(189), Integer(376), Integer(919)]
      >>> Superincreasing(L).is_superincreasing()
      True
      >>> Superincreasing().is_superincreasing([Integer(1),Integer(3),Integer(5),Integer(7)])
      False
      >>> seq = Superincreasing(); seq
      An empty sequence.
      >>> seq = Superincreasing([Integer(1), Integer(3), Integer(6)]); seq
      Super-increasing sequence of length 3
      >>> seq = Superincreasing([Integer(1), Integer(2), Integer(5), Integer(21), Integer(69), Integer(189), Integer(376), Integer(919)]); seq
      Super-increasing sequence of length 8

  is_superincreasing(seq=None)

    Determine whether or not `seq` is super-increasing.

    If `seq=None` then determine whether or not `self` is super-increasing.

    Let \(L = (a_1, a_2, a_3, \dots, a_n)\) be a non-empty sequence of nonnegative integers. Then \(L\) is said to be super-increasing if each \(a_i\) is strictly greater than the sum of all previous values. That is, for each \(a_i \in L\) the sequence \(L\) must satisfy the property

    \[a_i > \sum_{k=1}^{i-1} a_k\]

    in order to be called a super-increasing sequence, where \(|L| \geq 2\). If \(L\) has exactly one element, then it is also defined to be a super-increasing sequence.

    INPUT:

    - `seq` – (default: `None`) a sequence to test

    OUTPUT:

    - If `seq` is `None`, then test `self` to determine whether or not it is super-increasing. In that case, return `True` if `self` is super-increasing; `False` otherwise.

    - If `seq` is not `None`, then test `seq` to determine whether or not it is super-increasing. Return `True` if `seq` is super-increasing; `False` otherwise.

    EXAMPLES:

    By definition, an empty sequence is not super-increasing:

    Sage

        sage: from sage.numerical.knapsack import Superincreasing
        sage: Superincreasing().is_superincreasing([])
        False
        sage: Superincreasing().is_superincreasing()
        False
        sage: Superincreasing().is_superincreasing(tuple())
        False
        sage: Superincreasing().is_superincreasing(())
        False

    Python

        >>> from sage.all import *
        >>> from sage.numerical.knapsack import Superincreasing
        >>> Superincreasing().is_superincreasing([])
        False
        >>> Superincreasing().is_superincreasing()
        False
        >>> Superincreasing().is_superincreasing(tuple())
        False
        >>> Superincreasing().is_superincreasing(())
        False

    But here is an example of a super-increasing sequence:

    Sage

        sage: L = [1, 2, 5, 21, 69, 189, 376, 919]
        sage: Superincreasing(L).is_superincreasing()
        True
        sage: L = (1, 2, 5, 21, 69, 189, 376, 919)
        sage: Superincreasing(L).is_superincreasing()
        True

    [Python]

        >>> from sage.all import *
        >>> L = [Integer(1), Integer(2), Integer(5), Integer(21), Integer(69), Integer(189), Integer(376), Integer(919)]
        >>> Superincreasing(L).is_superincreasing()
        True
        >>> L = (Integer(1), Integer(2), Integer(5), Integer(21), Integer(69), Integer(189), Integer(376), Integer(919))
        >>> Superincreasing(L).is_superincreasing()
        True

    A super-increasing sequence can have zero as one of its elements:

    Sage

        sage: L = [0, 1, 2, 4]
        sage: Superincreasing(L).is_superincreasing()
        True

    Python

        >>> from sage.all import *
        >>> L = [Integer(0), Integer(1), Integer(2), Integer(4)]
        >>> Superincreasing(L).is_superincreasing()
        True

    A super-increasing sequence can be of length 1:

    Sage

        sage: Superincreasing([randint(0, 100)]).is_superincreasing()
        True

    [Python]

        >>> from sage.all import *
        >>> Superincreasing([randint(Integer(0), Integer(100))]).is_superincreasing()
        True

  largest_less_than(N)

    Return the largest integer in the sequence `self` that is less than or equal to `N`.

    This function narrows down the candidate solution using a binary trim, similar to the way binary search halves the sequence at each iteration.

    INPUT:

    - `N` – integer; the target value to search for

    OUTPUT:

    The largest integer in `self` that is less than or equal to `N`. If no solution exists, then return `None`.

    EXAMPLES:

    When a solution is found, return it:

    Sage

        sage: from sage.numerical.knapsack import Superincreasing
        sage: L = [2, 3, 7, 25, 67, 179, 356, 819]
        sage: Superincreasing(L).largest_less_than(207)
        179
        sage: L = (2, 3, 7, 25, 67, 179, 356, 819)
        sage: Superincreasing(L).largest_less_than(2)
        2

    Python

        >>> from sage.all import *
        >>> from sage.numerical.knapsack import Superincreasing
        >>> L = [Integer(2), Integer(3), Integer(7), Integer(25), Integer(67), Integer(179), Integer(356), Integer(819)]
        >>> Superincreasing(L).largest_less_than(Integer(207))
        179
        >>> L = (Integer(2), Integer(3), Integer(7), Integer(25), Integer(67), Integer(179), Integer(356), Integer(819))
        >>> Superincreasing(L).largest_less_than(Integer(2))
        2

    But if no solution exists, return `None`:

    Sage

        sage: L = [2, 3, 7, 25, 67, 179, 356, 819]
        sage: Superincreasing(L).largest_less_than(-1) is None
        True

    [Python]

        >>> from sage.all import *
        >>> L = [Integer(2), Integer(3), Integer(7), Integer(25), Integer(67), Integer(179), Integer(356), Integer(819)]
        >>> Superincreasing(L).largest_less_than(-Integer(1)) is None
        True

  subset_sum(N)

    Solving the subset sum problem for a super-increasing sequence.

    Let \(S = (s_1, s_2, s_3, \dots, s_n)\) be a non-empty sequence of nonnegative integers, and let \(N \in \ZZ\) be nonnegative. The subset sum problem asks for a subset \(A \subseteq S\) all of whose elements sum to \(N\). This method specializes the subset sum problem to the case of super-increasing sequences. If a solution exists, then it is also a super-increasing sequence.

    Note

    This method only solves the subset sum problem for super-increasing sequences. In general, solving the subset sum problem for an arbitrary sequence is known to be computationally hard.

    INPUT:

    - `N` – nonnegative integer

    OUTPUT:

    - A non-empty subset of `self` whose elements sum to `N`. This subset is also a super-increasing sequence. If no such subset exists, then return the empty list.

    ALGORITHM:

    The algorithm used is adapted from page 355 of [HPS2008].

    EXAMPLES:

    Solving the subset sum problem for a super-increasing sequence and target sum:

    Sage

        sage: from sage.numerical.knapsack import Superincreasing
        sage: L = [1, 2, 5, 21, 69, 189, 376, 919]
        sage: Superincreasing(L).subset_sum(98)
        [69, 21, 5, 2, 1]

    Python

        >>> from sage.all import *
        >>> from sage.numerical.knapsack import Superincreasing
        >>> L = [Integer(1), Integer(2), Integer(5), Integer(21), Integer(69), Integer(189), Integer(376), Integer(919)]
        >>> Superincreasing(L).subset_sum(Integer(98))
        [69, 21, 5, 2, 1]

sage.numerical.knapsack.knapsack(seq, binary, max=True, value_only=1, solver=False, verbose=None, integrality_tolerance=0)

  Solve the knapsack problem.

  For more information on the knapsack problem, see the documentation of the `knapsack module` or the Wikipedia article Knapsack_problem.

  INPUT:

  - `seq` – two different possible types:

    - A sequence of tuples `(weight, value, something1, something2, ...)`. Note that only the first two coordinates (`weight` and `values`) will be taken into account. The rest (if any) will be ignored. This can be useful if you need to attach some information to the items.

    - A sequence of reals (a value of 1 is assumed).

  - `binary` – when set to `True`, an item can be taken 0 or 1 time When set to `False`, an item can be taken any amount of times (while staying integer and positive).

  - `max` – maximum admissible weight

  - `value_only` – when set to `True`, only the maximum useful value is returned. When set to `False`, both the maximum useful value and an assignment are returned.

  - `solver` – (default: `None`) specify a Mixed Integer Linear Programming (MILP) solver to be used. If set to `None`, the default one is used. For more information on MILP solvers and which default solver is used, see the method `solve` of the class `MixedIntegerLinearProgram`.

  - `verbose` – integer (default: 0); sets the level of verbosity. Set to 0 by default, which means quiet.

  - `integrality_tolerance` – parameter for use with MILP solvers over an inexact base ring; see `MixedIntegerLinearProgram.get_values()`

  OUTPUT:

  If `value_only` is set to `True`, only the maximum useful value is returned. Else (the default), the function returns a pair `[value,list]`, where `list` can be of two types according to the type of `seq`:

  - The list of tuples \((w_i, u_i, ...)\) occurring in the solution.

  - A list of reals where each real is repeated the number of times it is taken into the solution.

  EXAMPLES:

  If your knapsack problem is composed of three items `(weight, value)` defined by `(1,2), (1.5,1), (0.5,3)`, and a bag of maximum weight \(2\), you can easily solve it this way:

  Sage

      sage: from sage.numerical.knapsack import knapsack
      sage: knapsack( [(1,2), (1.5,1), (0.5,3)], max=2)
      [5.0, [(1, 2), (0.500000000000000, 3)]]

      sage: knapsack( [(1,2), (1.5,1), (0.5,3)], max=2, value_only=True)
      5.0

  Python

      >>> from sage.all import *
      >>> from sage.numerical.knapsack import knapsack
      >>> knapsack( [(Integer(1),Integer(2)), (RealNumber('1.5'),Integer(1)), (RealNumber('0.5'),Integer(3))], max=Integer(2))
      [5.0, [(1, 2), (0.500000000000000, 3)]]

      >>> knapsack( [(Integer(1),Integer(2)), (RealNumber('1.5'),Integer(1)), (RealNumber('0.5'),Integer(3))], max=Integer(2), value_only=True)
      5.0

  Besides weight and value, you may attach any data to the items:

  Sage

      sage: from sage.numerical.knapsack import knapsack
      sage: knapsack( [(1, 2, 'spam'), (0.5, 3, 'a', 'lot')])
      [3.0, [(0.500000000000000, 3, 'a', 'lot')]]

  [Python]

      >>> from sage.all import *
      >>> from sage.numerical.knapsack import knapsack
      >>> knapsack( [(Integer(1), Integer(2), 'spam'), (RealNumber('0.5'), Integer(3), 'a', 'lot')])
      [3.0, [(0.500000000000000, 3, 'a', 'lot')]]

  In the case where all the values (usefulness) of the items are equal to one, you do not need embarrass yourself with the second values, and you can just type for items \((1,1), (1.5,1), (0.5,1)\) the command:

  Sage

      sage: from sage.numerical.knapsack import knapsack
      sage: knapsack([1,1.5,0.5], max=2, value_only=True)
      2.0

  Python

      >>> from sage.all import *
      >>> from sage.numerical.knapsack import knapsack
      >>> knapsack([Integer(1),RealNumber('1.5'),RealNumber('0.5')], max=Integer(2), value_only=True)
      2.0