common lisp: value 16777216 is not of the expected type (UNSIGNED-BYTE 24)

Question

I'm using Clozure-CL on windows 7. I wrote a simple “image” class (well, struct) for a test.

(defmacro -> (struct slot) `(slot-value ,struct ,slot))

(defstruct 
  (image 
    (:constructor make-image (&key width height (bytes-per-pixel 4) 
        (pixels (make-array (* bytes-per-pixel (* width height))
                                :element-type '(unsigned-byte 8)))))
    (:print-function (lambda (img s k)
                       (declare (ignore k)) 
               (format s "image(~ax~a@~abpp)"
                               (-> img 'width) (-> img 'height)
                               (* 8 (-> img 'bytes-per-pixel))))))
  (width 0 :type '(unsigned-byte 32))
  (height 0 :type '(unsigned-byte 32))
  (bytes-per-pixel 4 :type '(unsigned-byte 32))
  (pixels nil))

Unfortunately, when I try to do this:

(make-image :width 2048 :height 2048)

I get this error:

value 16777216 is not of the expected type (UNSIGNED-BYTE 24).
   [Condition of type TYPE-ERROR]

Here's stack trace (copy-pasted from slimv):

  0: (CCL::MAKE-UARRAY-1 207 16777216 NIL NIL NIL NIL NIL NIL NIL 16777216)
  1: (MAKE-IMAGE :WIDTH 2048 :HEIGHT 2048 :BYTES-PER-PIXEL 4 :PIXELS 16777216)
  2: (CCL::CALL-CHECK-REGS MAKE-IMAGE :WIDTH 2048 :HEIGHT 2048)
  3: (CCL::CHEAP-EVAL (MAKE-IMAGE :WIDTH 2048 :HEIGHT 2048))
  4: (SWANK::EVAL-REGION "(make-image :width 2048 :height 2048)\n")
  5: ((:INTERNAL SWANK::REPL-EVAL))
  6: (SWANK::TRACK-PACKAGE #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::REPL-EVAL) #x19D56EAE>)
  7: (SWANK::CALL-WITH-RETRY-RESTART "Retry SLIME REPL evaluation request." #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::REPL-EVAL) #x19D56EFE>)
  8: (SWANK::CALL-WITH-BUFFER-SYNTAX NIL #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::REPL-EVAL) #x19D56F26>)
  9: (SWANK::REPL-EVAL "(make-image :width 2048 :height 2048)\n")
 10: (CCL::CALL-CHECK-REGS SWANK:LISTENER-EVAL "(make-image :width 2048 :height 2048)\n")
 11: (CCL::CHEAP-EVAL (SWANK:LISTENER-EVAL "(make-image :width 2048 :height 2048)\n"))
 12: (SWANK:EVAL-FOR-EMACS (SWANK:LISTENER-EVAL "(make-image :width 2048 :height 2048)\n") "GAME" 135)
 13: (SWANK::PROCESS-REQUESTS NIL)
 14: ((:INTERNAL SWANK::HANDLE-REQUESTS))
 15: ((:INTERNAL SWANK::HANDLE-REQUESTS))
 16: (SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK #<Compiled-function SWANK:SWANK-DEBUGGER-HOOK #x186F1EF6> #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::HANDLE-REQUESTS) #x1882D666>)
 17: (SWANK::CALL-WITH-BINDINGS ((*STANDARD-OUTPUT* . #<SWANK-BACKEND::SLIME-OUTPUT-STREAM #x188247CE>) (*STANDARD-INPUT* . #<SWANK-BACKEND::SLIME-INPUT-STREAM #x188249DE>) ..))) #<CCL:COMPILED-LEXICAL-CLO..
 18: (SWANK::HANDLE-REQUESTS #<MULTITHREADED-CONNECTION  #x1880CB0E> NIL)
 19: (CCL::RUN-PROCESS-INITIAL-FORM #<PROCESS repl-thread(12) [Active] #x18824C8E> (#<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL CCL::%PROCESS-RUN-FUNCTION) #x18824B46>))

As I understand it, Clozure-CL uses some internal function to make byte arrays, and this function for some reason expects 24-bit integer arguments for array size and because 16777216 is 2²⁴, it won't fit into 24-bit integer. I'd like to remove this limitation. How can I fix this problem?

Edit

I checked documentation and found out that on 32-bit Clozure-CL the maximum array size limit (array-total-size-limit) is (expt 2 24). The 64-bit version has a much larger limit, but still, can I do something about it?

Answer 1

As you already found out, that's a limitation by the 32bit Clozure CL version. The 64bit Clozure CL version has a larger limit.

On my Mac:

? array-total-size-limit
72057594037927936

What can you do:

• use several smaller arrays and hide it behind some interface. Painful.
• use the FFI to allocate arrays on the C side. A bit painful. Make sure you have some type/bounds checking and get the memory management right.
• use the 64bit version. Maybe that's an option for you?

The implementation limits of the 32bit version are not easy to change, if at all.