Mirrors
/
php
mirror of https://github.com/php/php-src
3
0
Fork 0
Code Releases Activity
Browse Source

Pass opline as argument to opcode handlers in CALL VM 

This changes the signature of opcode handlers in the CALL VM so that the opline
is passed directly via arguments. This reduces the number of memory operations
on EX(opline), and makes the CALL VM considerably faster.

Additionally, this unifies the CALL and HYBRID VMs a bit, as EX(opline) is now
handled in the same way in both VMs.

This is a part of GH-17849.

Currently we have two VMs:

 * HYBRID: Used when compiling with GCC. execute_data and opline are global
   register variables
 * CALL: Used when compiling with something else. execute_data is passed as
   opcode handler arg, but opline is passed via execute_data->opline
   (EX(opline)).

The Call VM looks like this:

    while (1) {
        ret = execute_data->opline->handler(execute_data);
        if (UNEXPECTED(ret != 0)) {
            if (ret > 0) { // returned by ZEND_VM_ENTER() / ZEND_VM_LEAVE()
                execute_data = EG(current_execute_data);
            } else {       // returned by ZEND_VM_RETURN()
                return;
            }
        }
    }

    // example op handler
    int ZEND_INIT_FCALL_SPEC_CONST_HANDLER(zend_execute_data *execute_data) {
        // load opline
        const zend_op *opline = execute_data->opline;

        // instruction execution

        // dispatch
        // ZEND_VM_NEXT_OPCODE():
        execute_data->opline++;
        return 0; // ZEND_VM_CONTINUE()
    }

Opcode handlers return a positive value to signal that the loop must load a
new execute_data from EG(current_execute_data), typically when entering
or leaving a function.

Here I make the following changes:

 * Pass opline as opcode handler argument
 * Return next opline from opcode handlers
 * ZEND_VM_ENTER / ZEND_VM_LEAVE return opline|(1<<0) to signal that
   execute_data must be reloaded from EG(current_execute_data)

This gives us:

    while (1) {
        opline = opline->handler(execute_data, opline);
        if (UNEXPECTED((uintptr_t) opline & ZEND_VM_ENTER_BIT) {
            opline = opline & ~ZEND_VM_ENTER_BIT;
            if (opline != 0) { // ZEND_VM_ENTER() / ZEND_VM_LEAVE()
                execute_data = EG(current_execute_data);
            } else {           // ZEND_VM_RETURN()
                return;
            }
        }
    }

    // example op handler
    const zend_op * ZEND_INIT_FCALL_SPEC_CONST_HANDLER(zend_execute_data *execute_data, const zend_op *opline) {
        // opline already loaded

        // instruction execution

        // dispatch
        // ZEND_VM_NEXT_OPCODE():
        return ++opline;
    }

bench.php is 23% faster on Linux / x86_64, 18% faster on MacOS / M1.

Symfony Demo is 2.8% faster.

When using the HYBRID VM, JIT'ed code stores execute_data/opline in two fixed
callee-saved registers and rarely touches EX(opline), just like the VM.

Since the registers are callee-saved, the JIT'ed code doesn't have to
save them before calling other functions, and can assume they always
contain execute_data/opline. The code also avoids saving/restoring them in
prologue/epilogue, as execute_ex takes care of that (JIT'ed code is called
exclusively from there).

The CALL VM can now use a fixed register for execute_data/opline as well, but
we can't rely on execute_ex to save the registers for us as it may use these
registers itself. So we have to save/restore the two registers in JIT'ed code
prologue/epilogue.

Closes GH-17952
pull/18338/head
Arnaud Le Blanc 8 months ago
parent
49891d89fc
commit
76d7c616bb
Failed to extract signature
11 changed files with 457 additions and 481 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 3
      Zend/zend_vm_def.h
  2. 372
      Zend/zend_vm_execute.h
  3. 2
      Zend/zend_vm_execute.skl
  4. 92
      Zend/zend_vm_gen.php
  5. 2
      Zend/zend_vm_opcodes.h
  6. 27
      ext/opcache/jit/zend_jit.c
  7. 38
      ext/opcache/jit/zend_jit_internal.h
  8. 292
      ext/opcache/jit/zend_jit_ir.c
  9. 7
      ext/opcache/jit/zend_jit_trace.c
  10. 99
      ext/opcache/jit/zend_jit_vm_helpers.c
  11. 4
      sapi/phpdbg/tests/watch_006.phpt

3
Zend/zend_vm_def.h
View File

@ -5656,11 +5656,10 @@ ZEND_VM_HOT_HANDLER(199, ZEND_CHECK_UNDEF_ARGS, UNUSED, UNUSED)
ZEND_VM_COLD_HELPER(zend_missing_arg_helper, ANY, ANY)
{
#ifdef ZEND_VM_IP_GLOBAL_REG
USE_OPLINE
SAVE_OPLINE();
#endif
zend_missing_arg_error(execute_data);
HANDLE_EXCEPTION();
}

372
Zend/zend_vm_execute.h
File diff suppressed because it is too large
View File

2
Zend/zend_vm_execute.skl
View File

@ -1,3 +1,5 @@
#include "Zend/zend_vm_opcodes.h"
{%DEFINES%}
#if (ZEND_VM_KIND != ZEND_VM_KIND_CALL) && (ZEND_GCC_VERSION >= 4000) && !defined(__clang__)

92
Zend/zend_vm_gen.php
View File

@ -818,7 +818,7 @@ function gen_code($f, $spec, $kind, $code, $op1, $op2, $name, $extra_spec=null)
if (isset($matches[2])) {
// extra args
$args = substr(preg_replace("/,\s*[A-Za-z0-9_]*\s*,\s*([^,)\s]*)\s*/", ", $1", $matches[2]), 2);
return "ZEND_VM_TAIL_CALL(" . helper_name($matches[1], $spec, $op1, $op2, $extra_spec) . "(" . $args. " ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_CC))";
return "ZEND_VM_TAIL_CALL(" . helper_name($matches[1], $spec, $op1, $op2, $extra_spec) . "(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX " . $args . "))";
}
return "ZEND_VM_TAIL_CALL(" . helper_name($matches[1], $spec, $op1, $op2, $extra_spec) . "(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU))";
}
@ -852,7 +852,7 @@ function gen_code($f, $spec, $kind, $code, $op1, $op2, $name, $extra_spec=null)
if (isset($matches[2])) {
// extra args
$args = substr(preg_replace("/,\s*[A-Za-z0-9_]*\s*,\s*([^,)\s]*)\s*/", ", $1", $matches[2]), 2);
return "ZEND_VM_TAIL_CALL(" . helper_name($matches[1], $spec, $op1, $op2, $extra_spec) . "(" . $args. " ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_CC))";
return "ZEND_VM_TAIL_CALL(" . helper_name($matches[1], $spec, $op1, $op2, $extra_spec) . "(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX " . $args . "))";
}
return "ZEND_VM_TAIL_CALL(" . helper_name($matches[1], $spec, $op1, $op2, $extra_spec) . "(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU))";
}
@ -1164,7 +1164,7 @@ function gen_helper($f, $spec, $kind, $name, $op1, $op2, $param, $code, $lineno,
out($f, "static$zend_attributes ZEND_OPCODE_HANDLER_RET$zend_fastcall $spec_name(ZEND_OPCODE_HANDLER_ARGS)\n");
} else {
// Helper with parameter
out($f, "static$zend_attributes ZEND_OPCODE_HANDLER_RET$zend_fastcall $spec_name($param ZEND_OPCODE_HANDLER_ARGS_DC)\n");
out($f, "static$zend_attributes ZEND_OPCODE_HANDLER_RET$zend_fastcall $spec_name(ZEND_OPCODE_HANDLER_ARGS_EX $param)\n");
}
break;
case ZEND_VM_KIND_SWITCH:
@ -1870,13 +1870,13 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
out($f,"#ifdef ZEND_VM_FP_GLOBAL_REG\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS void\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_DC\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_CC\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_EX\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX\n");
out($f,"#else\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS zend_execute_data *execute_data\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU execute_data\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_DC , ZEND_OPCODE_HANDLER_ARGS\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_CC , ZEND_OPCODE_HANDLER_ARGS_PASSTHRU\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS zend_execute_data *execute_data, const zend_op *opline\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU execute_data, opline\n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_EX ZEND_OPCODE_HANDLER_ARGS, \n");
out($f,"# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX ZEND_OPCODE_HANDLER_ARGS_PASSTHRU, \n");
out($f,"#endif\n");
out($f,"\n");
out($f,"#if defined(ZEND_VM_FP_GLOBAL_REG) && defined(ZEND_VM_IP_GLOBAL_REG)\n");
@ -1902,10 +1902,10 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
out($f,"# define ZEND_VM_COLD ZEND_COLD ZEND_OPT_SIZE\n");
}
out($f,"#else\n");
out($f,"# define ZEND_OPCODE_HANDLER_RET int\n");
out($f,"# define ZEND_OPCODE_HANDLER_RET const zend_op *\n");
out($f,"# define ZEND_VM_TAIL_CALL(call) return call\n");
out($f,"# define ZEND_VM_CONTINUE() return 0\n");
out($f,"# define ZEND_VM_RETURN() return -1\n");
out($f,"# define ZEND_VM_CONTINUE() return opline\n");
out($f,"# define ZEND_VM_RETURN() return (const zend_op*)ZEND_VM_ENTER_BIT\n");
if ($kind == ZEND_VM_KIND_HYBRID) {
out($f,"# define ZEND_VM_HOT\n");
}
@ -1914,8 +1914,8 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
out($f,"\n");
out($f,"typedef ZEND_OPCODE_HANDLER_RET (ZEND_FASTCALL *opcode_handler_t) (ZEND_OPCODE_HANDLER_ARGS);\n");
out($f,"\n");
out($f,"#define DCL_OPLINE\n");
out($f,"#ifdef ZEND_VM_IP_GLOBAL_REG\n");
out($f,"# define DCL_OPLINE\n");
out($f,"# define OPLINE opline\n");
out($f,"# define USE_OPLINE\n");
out($f,"# define LOAD_OPLINE() opline = EX(opline)\n");
@ -1924,12 +1924,13 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
out($f,"# define SAVE_OPLINE() EX(opline) = opline\n");
out($f,"# define SAVE_OPLINE_EX() SAVE_OPLINE()\n");
out($f,"#else\n");
out($f,"# define OPLINE EX(opline)\n");
out($f,"# define USE_OPLINE const zend_op *opline = EX(opline);\n");
out($f,"# define LOAD_OPLINE()\n");
out($f,"# define LOAD_OPLINE_EX()\n");
out($f,"# define LOAD_NEXT_OPLINE() ZEND_VM_INC_OPCODE()\n");
out($f,"# define SAVE_OPLINE()\n");
out($f,"# define DCL_OPLINE const zend_op *opline;\n");
out($f,"# define OPLINE opline\n");
out($f,"# define USE_OPLINE\n");
out($f,"# define LOAD_OPLINE() opline = EX(opline)\n");
out($f,"# define LOAD_OPLINE_EX() opline = EX(opline)\n");
out($f,"# define LOAD_NEXT_OPLINE() opline = EX(opline) + 1\n");
out($f,"# define SAVE_OPLINE() EX(opline) = opline\n");
out($f,"# define SAVE_OPLINE_EX()\n");
out($f,"#endif\n");
out($f,"#define HANDLE_EXCEPTION() ZEND_ASSERT(EG(exception)); LOAD_OPLINE(); ZEND_VM_CONTINUE()\n");
@ -1943,9 +1944,10 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
out($f,"# define ZEND_VM_ENTER() opline = EG(current_execute_data)->opline; ZEND_VM_ENTER_EX()\n");
out($f,"# define ZEND_VM_LEAVE() return 2\n");
out($f,"#else\n");
out($f,"# define ZEND_VM_ENTER_EX() return 1\n");
out($f,"# define ZEND_VM_ENTER() return 1\n");
out($f,"# define ZEND_VM_LEAVE() return 2\n");
out($f,"# define ZEND_VM_ENTER_BIT 1ULL\n");
out($f,"# define ZEND_VM_ENTER_EX() return (zend_op*)((uintptr_t)opline | ZEND_VM_ENTER_BIT)\n");
out($f,"# define ZEND_VM_ENTER() execute_data = EG(current_execute_data); LOAD_OPLINE(); ZEND_VM_ENTER_EX()\n");
out($f,"# define ZEND_VM_LEAVE() return (zend_op*)((uintptr_t)opline | ZEND_VM_ENTER_BIT)\n");
out($f,"#endif\n");
out($f,"#define ZEND_VM_INTERRUPT() ZEND_VM_TAIL_CALL(zend_interrupt_helper".($spec?"_SPEC":"")."(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU));\n");
out($f,"#define ZEND_VM_LOOP_INTERRUPT() zend_interrupt_helper".($spec?"_SPEC":"")."(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU);\n");
@ -2119,12 +2121,7 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
}
break;
case "ZEND_VM_CONTINUE_LABEL":
if ($kind == ZEND_VM_KIND_CALL || $kind == ZEND_VM_KIND_HYBRID) {
// Only SWITCH dispatch method use it
out($f,"#if !defined(ZEND_VM_FP_GLOBAL_REG) || !defined(ZEND_VM_IP_GLOBAL_REG)\n");
out($f,$m[1]."\tint ret;".$m[3]."\n");
out($f,"#endif\n");
} else if ($kind == ZEND_VM_KIND_SWITCH) {
if ($kind == ZEND_VM_KIND_SWITCH) {
// Only SWITCH dispatch method use it
out($f,"zend_vm_continue:".$m[3]."\n");
} else {
@ -2143,16 +2140,17 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
case ZEND_VM_KIND_HYBRID:
out($f,"#if (ZEND_VM_KIND == ZEND_VM_KIND_HYBRID)\n");
out($f, $m[1]."HYBRID_SWITCH()".$m[3]."\n");
out($f,"#else\n");
out($f,"#else /* ZEND_VM_KIND != ZEND_VM_KIND_HYBRID */\n");
case ZEND_VM_KIND_CALL:
out($f,"#if defined(ZEND_VM_FP_GLOBAL_REG) && defined(ZEND_VM_IP_GLOBAL_REG)\n");
out($f, $m[1]."((opcode_handler_t)OPLINE->handler)(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU);\n");
out($f, $m[1]."if (UNEXPECTED(!OPLINE))".$m[3]."\n");
out($f,"#else\n");
out($f, $m[1]."if (UNEXPECTED((ret = ((opcode_handler_t)OPLINE->handler)(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU)) != 0))".$m[3]."\n");
out($f, $m[1]."opline = ((opcode_handler_t)opline->handler)(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU);\n");
out($f, $m[1]."if (UNEXPECTED(((uintptr_t)opline & ZEND_VM_ENTER_BIT)))".$m[3]."\n");
out($f,"#endif\n");
if ($kind == ZEND_VM_KIND_HYBRID) {
out($f,"#endif\n");
out($f,"#endif /* ZEND_VM_KIND != ZEND_VM_KIND_HYBRID */\n");
}
break;
}
@ -2168,7 +2166,7 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
if ($kind == ZEND_VM_KIND_CALL || $kind == ZEND_VM_KIND_HYBRID) {
// Executor is defined as a set of functions
if ($kind == ZEND_VM_KIND_HYBRID) {
out($f,"#else\n");
out($f,"#else /* ZEND_VM_KIND != ZEND_VM_KIND_HYBRID */\n");
}
out($f,
"#ifdef ZEND_VM_FP_GLOBAL_REG\n" .
@ -2178,18 +2176,19 @@ function gen_executor($f, $skl, $spec, $kind, $executor_name, $initializer_name)
"# endif\n" .
$m[1]."return;\n" .
"#else\n" .
$m[1]."if (EXPECTED(ret > 0)) {\n" .
$m[1]."opline = (const zend_op*)((uintptr_t)opline & ~ZEND_VM_ENTER_BIT);\n".
$m[1]."if (EXPECTED(opline != NULL)) {\n" .
$m[1]."\texecute_data = EG(current_execute_data);\n".
$m[1]."\tZEND_VM_LOOP_INTERRUPT_CHECK();\n".
$m[1]."} else {\n" .
"# ifdef ZEND_VM_IP_GLOBAL_REG\n" .
$m[1]."\topline = vm_stack_data.orig_opline;\n" .
"# endif\n".
"# endif\n" .
$m[1]."\treturn;\n".
$m[1]."}\n".
"#endif\n");
if ($kind == ZEND_VM_KIND_HYBRID) {
out($f,"#endif\n");
out($f,"#endif /* ZEND_VM_KIND != ZEND_VM_KIND_HYBRID */\n");
}
}
break;
@ -2335,6 +2334,8 @@ function gen_vm_opcodes_header(
): string {
$str = HEADER_TEXT;
$str .= "#ifndef ZEND_VM_OPCODES_H\n#define ZEND_VM_OPCODES_H\n\n";
$str .= "#include \"Zend/zend_portability.h\"\n";
$str .= "\n";
$str .= "#define ZEND_VM_SPEC\t\t" . ZEND_VM_SPEC . "\n";
$str .= "#define ZEND_VM_LINES\t\t" . ZEND_VM_LINES . "\n";
$str .= "#define ZEND_VM_KIND_CALL\t" . ZEND_VM_KIND_CALL . "\n";
@ -2968,6 +2969,7 @@ function gen_vm($def, $skel) {
out($f, "\topcode_handler_t handler;\n");
out($f,"#endif\n");
}
out($f, "\tDCL_OPLINE;\n");
out($f, "\tint ret;\n");
out($f, "#ifdef ZEND_VM_IP_GLOBAL_REG\n");
out($f, "\tconst zend_op *orig_opline = opline;\n");
@ -3001,8 +3003,24 @@ function gen_vm($def, $skel) {
out($f, "\t\tret = -1;\n");
out($f, "\t}\n");
out($f, "#else\n");
out($f, "\tret = ((opcode_handler_t)OPLINE->handler)(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU);\n");
out($f, "\tSAVE_OPLINE();\n");
out($f, "\topline = ((opcode_handler_t)OPLINE->handler)(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU);\n");
out($f, "if (UNEXPECTED(((uintptr_t)opline & ZEND_VM_ENTER_BIT))) {\n");
out($f, "\t\topline = (const zend_op*)((uintptr_t)opline & ~ZEND_VM_ENTER_BIT);\n");
out($f, "\t\tif (EXPECTED(opline)) {\n");
out($f, "\t\t\t/* ZEND_VM_ENTER() or ZEND_VM_LEAVE() */\n");
out($f, "\t\t\tret = EG(current_execute_data) != ex ? (int)(EG(current_execute_data)->prev_execute_data != ex) + 1 : 0;\n");
out($f, "\t\t\texecute_data = EG(current_execute_data);\n");
out($f, "\t\t\tSAVE_OPLINE();\n");
out($f, "\t\t} else {\n");
out($f, "\t\t\t/* ZEND_VM_RETURN() */\n");
out($f, "\t\t\tret = -1;\n");
out($f, "\t\t}\n");
out($f, "\t} else {\n");
out($f, "\t\t/* ZEND_VM_CONTINUE() */\n");
out($f, "\t\tSAVE_OPLINE();\n");
out($f, "\t\tret = 0;\n");
out($f, "\t}\n");
out($f, "#endif\n");
out($f, "#ifdef ZEND_VM_FP_GLOBAL_REG\n");
out($f, "\texecute_data = orig_execute_data;\n");

2
Zend/zend_vm_opcodes.h
View File

@ -21,6 +21,8 @@
#ifndef ZEND_VM_OPCODES_H
#define ZEND_VM_OPCODES_H
#include "Zend/zend_portability.h"
#define ZEND_VM_SPEC 1
#define ZEND_VM_LINES 0
#define ZEND_VM_KIND_CALL 1

27
ext/opcache/jit/zend_jit.c
View File

@ -98,7 +98,7 @@ static const void *zend_jit_func_trace_counter_handler = NULL;
static const void *zend_jit_ret_trace_counter_handler = NULL;
static const void *zend_jit_loop_trace_counter_handler = NULL;
static int ZEND_FASTCALL zend_runtime_jit(void);
static ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_runtime_jit(ZEND_OPCODE_HANDLER_ARGS);
static int zend_jit_trace_op_len(const zend_op *opline);
static int zend_jit_trace_may_exit(const zend_op_array *op_array, const zend_op *opline);
@ -2871,7 +2871,7 @@ static int zend_jit(const zend_op_array *op_array, zend_ssa *ssa, const zend_op
if (GCC_GLOBAL_REGS) {
ir_TAILCALL(IR_VOID, ir_LOAD_A(jit_IP(jit)));
} else {
ir_RETURN(ir_CONST_I32(1)); /* ZEND_VM_ENTER */
zend_jit_vm_enter(jit, jit_IP(jit));
}
ir_IF_TRUE(if_hook_enter);
}
@ -3074,11 +3074,18 @@ jit_failure:
}
/* Run-time JIT handler */
static int ZEND_FASTCALL zend_runtime_jit(void)
static ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_runtime_jit(ZEND_OPCODE_HANDLER_ARGS)
{
zend_execute_data *execute_data = EG(current_execute_data);
#if GCC_GLOBAL_REGS
zend_execute_data *execute_data;
zend_op *opline;
#else
const zend_op *orig_opline = opline;
#endif
execute_data = EG(current_execute_data);
zend_op_array *op_array = &EX(func)->op_array;
zend_op *opline = op_array->opcodes;
opline = op_array->opcodes;
zend_jit_op_array_extension *jit_extension;
bool do_bailout = 0;
@ -3097,7 +3104,7 @@ static int ZEND_FASTCALL zend_runtime_jit(void)
}
}
jit_extension = (zend_jit_op_array_extension*)ZEND_FUNC_INFO(op_array);
opline->handler = jit_extension->orig_handler;
((zend_op*)opline)->handler = jit_extension->orig_handler;
/* perform real JIT for this function */
zend_real_jit_func(op_array, NULL, NULL, ZEND_JIT_ON_FIRST_EXEC);
@ -3116,7 +3123,11 @@ static int ZEND_FASTCALL zend_runtime_jit(void)
}
/* JIT-ed code is going to be called by VM */
return 0;
#if GCC_GLOBAL_REGS
return; // ZEND_VM_CONTINUE
#else
return orig_opline; // ZEND_VM_CONTINUE
#endif
}
void zend_jit_check_funcs(HashTable *function_table, bool is_method) {
@ -3171,9 +3182,7 @@ void ZEND_FASTCALL zend_jit_hot_func(zend_execute_data *execute_data, const zend
op_array->opcodes[i].handler = jit_extension->orig_handlers[i];
}
#ifdef HAVE_GCC_GLOBAL_REGS
EX(opline) = opline;
#endif
/* perform real JIT for this function */
zend_real_jit_func(op_array, NULL, opline, ZEND_JIT_ON_HOT_COUNTERS);

38
ext/opcache/jit/zend_jit_internal.h
View File

@ -21,6 +21,12 @@
#ifndef ZEND_JIT_INTERNAL_H
#define ZEND_JIT_INTERNAL_H
#include "Zend/zend_types.h"
#include "Zend/zend_compile.h"
#include "Zend/zend_constants.h"
#include "Zend/Optimizer/zend_func_info.h"
#include "Zend/Optimizer/zend_call_graph.h"
/* Address Encoding */
typedef uintptr_t zend_jit_addr;
@ -183,17 +189,18 @@ extern const zend_op *zend_jit_halt_op;
# define ZEND_OPCODE_HANDLER_RET void
# define ZEND_OPCODE_HANDLER_ARGS EXECUTE_DATA_D
# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU
# define ZEND_OPCODE_HANDLER_ARGS_DC
# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_CC
# define ZEND_OPCODE_HANDLER_ARGS_EX
# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX
# define ZEND_OPCODE_RETURN() return
# define ZEND_OPCODE_TAIL_CALL(handler) do { \
handler(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU); \
return; \
} while(0)
# define ZEND_OPCODE_TAIL_CALL_EX(handler, arg) do { \
handler(arg ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_CC); \
handler(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX arg); \
return; \
} while(0)
# define ZEND_VM_ENTER_BIT 0
#else
# define EXECUTE_DATA_D zend_execute_data* execute_data
# define EXECUTE_DATA_C execute_data
@ -203,35 +210,36 @@ extern const zend_op *zend_jit_halt_op;
# define OPLINE_C opline
# define OPLINE_DC , OPLINE_D
# define OPLINE_CC , OPLINE_C
# define ZEND_OPCODE_HANDLER_RET int
# define ZEND_OPCODE_HANDLER_ARGS EXECUTE_DATA_D
# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU EXECUTE_DATA_C
# define ZEND_OPCODE_HANDLER_ARGS_DC EXECUTE_DATA_DC
# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_CC EXECUTE_DATA_CC
# define ZEND_OPCODE_RETURN() return 0
# define ZEND_OPCODE_HANDLER_RET const zend_op *
# define ZEND_OPCODE_HANDLER_ARGS EXECUTE_DATA_D OPLINE_DC
# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU EXECUTE_DATA_C OPLINE_CC
# define ZEND_OPCODE_HANDLER_ARGS_EX EXECUTE_DATA_D OPLINE_DC,
# define ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX EXECUTE_DATA_C OPLINE_CC,
# define ZEND_OPCODE_RETURN() return opline
# define ZEND_OPCODE_TAIL_CALL(handler) do { \
return handler(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU); \
} while(0)
# define ZEND_OPCODE_TAIL_CALL_EX(handler, arg) do { \
return handler(arg ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_CC); \
return handler(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX arg); \
} while(0)
# define ZEND_VM_ENTER_BIT 1ULL
#endif
/* VM handlers */
typedef ZEND_OPCODE_HANDLER_RET (ZEND_FASTCALL *zend_vm_opcode_handler_t)(ZEND_OPCODE_HANDLER_ARGS);
/* VM helpers */
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_nested_func_helper(uint32_t call_info EXECUTE_DATA_DC);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_top_func_helper(uint32_t call_info EXECUTE_DATA_DC);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_func_helper(EXECUTE_DATA_D);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_nested_func_helper(ZEND_OPCODE_HANDLER_ARGS_EX uint32_t call_info);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_top_func_helper(ZEND_OPCODE_HANDLER_ARGS_EX uint32_t call_info);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_func_helper(ZEND_OPCODE_HANDLER_ARGS);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_profile_helper(ZEND_OPCODE_HANDLER_ARGS);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_func_counter_helper(ZEND_OPCODE_HANDLER_ARGS);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_loop_counter_helper(ZEND_OPCODE_HANDLER_ARGS);
void ZEND_FASTCALL zend_jit_copy_extra_args_helper(EXECUTE_DATA_D);
void ZEND_FASTCALL zend_jit_copy_extra_args_helper_no_skip_recv(EXECUTE_DATA_D);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_copy_extra_args_helper(ZEND_OPCODE_HANDLER_ARGS);
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_copy_extra_args_helper_no_skip_recv(ZEND_OPCODE_HANDLER_ARGS);
bool ZEND_FASTCALL zend_jit_deprecated_helper(OPLINE_D);
bool ZEND_FASTCALL zend_jit_nodiscard_helper(OPLINE_D);
bool ZEND_FASTCALL zend_jit_deprecated_nodiscard_helper(OPLINE_D);

292
ext/opcache/jit/zend_jit_ir.c
View File

@ -911,18 +911,9 @@ static ir_ref jit_IP32(zend_jit_ctx *jit)
return ir_RLOAD_U32(ZREG_IP);
}
static void jit_LOAD_IP(zend_jit_ctx *jit, ir_ref ref)
{
if (GCC_GLOBAL_REGS) {
jit_STORE_IP(jit, ref);
} else {
ir_STORE(jit_EX(opline), ref);
}
}
static void jit_LOAD_IP_ADDR(zend_jit_ctx *jit, const zend_op *target)
{
jit_LOAD_IP(jit, ir_CONST_ADDR(target));
jit_STORE_IP(jit, ir_CONST_ADDR(target));
}
static void zend_jit_track_last_valid_opline(zend_jit_ctx *jit)
@ -1008,7 +999,6 @@ static int zend_jit_save_call_chain(zend_jit_ctx *jit, uint32_t call_level)
static int zend_jit_set_ip(zend_jit_ctx *jit, const zend_op *target)
{
ir_ref ref;
ir_ref addr = IR_UNUSED;
if (jit->delayed_call_level) {
if (!zend_jit_save_call_chain(jit, jit->delayed_call_level)) {
@ -1019,58 +1009,30 @@ static int zend_jit_set_ip(zend_jit_ctx *jit, const zend_op *target)
if (jit->last_valid_opline) {
zend_jit_use_last_valid_opline(jit);
if (jit->last_valid_opline != target) {
if (GCC_GLOBAL_REGS) {
ref = jit_IP(jit);
} else {
addr = jit_EX(opline);
ref = ir_LOAD_A(addr);
}
ref = jit_IP(jit);
if (target > jit->last_valid_opline) {
ref = ir_ADD_OFFSET(ref, (uintptr_t)target - (uintptr_t)jit->last_valid_opline);
} else {
ref = ir_SUB_A(ref, ir_CONST_ADDR((uintptr_t)jit->last_valid_opline - (uintptr_t)target));
}
if (GCC_GLOBAL_REGS) {
jit_STORE_IP(jit, ref);
} else {
ir_STORE(addr, ref);
}
jit_STORE_IP(jit, ref);
}
} else {
if (GCC_GLOBAL_REGS) {
jit_STORE_IP(jit, ir_CONST_ADDR(target));
} else {
ir_STORE(jit_EX(opline), ir_CONST_ADDR(target));
}
jit_STORE_IP(jit, ir_CONST_ADDR(target));
}
jit->reuse_ip = 0;
zend_jit_set_last_valid_opline(jit, target);
return 1;
}
static int zend_jit_set_ip_ex(zend_jit_ctx *jit, const zend_op *target, bool set_ip_reg)
{
if (!GCC_GLOBAL_REGS && set_ip_reg && !jit->last_valid_opline) {
/* Optimization to avoid duplicate constant load */
ir_STORE(jit_EX(opline), ir_HARD_COPY_A(ir_CONST_ADDR(target)));
return 1;
}
return zend_jit_set_ip(jit, target);
}
static void jit_SET_EX_OPLINE(zend_jit_ctx *jit, const zend_op *target)
{
if (jit->last_valid_opline == target) {
zend_jit_use_last_valid_opline(jit);
if (GCC_GLOBAL_REGS) {
// EX(opline) = opline
ir_STORE(jit_EX(opline), jit_IP(jit));
}
// EX(opline) = opline
ir_STORE(jit_EX(opline), jit_IP(jit));
} else {
ir_STORE(jit_EX(opline), ir_CONST_ADDR(target));
if (!GCC_GLOBAL_REGS) {
zend_jit_reset_last_valid_opline(jit);
}
}
}
@ -1912,6 +1874,18 @@ static void zend_jit_check_timeout(zend_jit_ctx *jit, const zend_op *opline, con
}
}
static void zend_jit_vm_enter(zend_jit_ctx *jit, ir_ref to_opline)
{
// ZEND_VM_ENTER()
ir_RETURN(ir_OR_A(to_opline, ir_CONST_ADDR(ZEND_VM_ENTER_BIT)));
}
static void zend_jit_vm_leave(zend_jit_ctx *jit, ir_ref to_opline)
{
// ZEND_VM_LEAVE()
ir_RETURN(ir_OR_A(to_opline, ir_CONST_ADDR(ZEND_VM_ENTER_BIT)));
}
/* stubs */
static int zend_jit_exception_handler_stub(zend_jit_ctx *jit)
@ -1929,14 +1903,8 @@ static int zend_jit_exception_handler_stub(zend_jit_ctx *jit)
if (GCC_GLOBAL_REGS) {
ir_TAILCALL(IR_VOID, ir_CONST_FUNC(handler));
} else {
ir_ref ref, if_negative;
ref = ir_CALL_1(IR_I32, ir_CONST_FC_FUNC(handler), jit_FP(jit));
if_negative = ir_IF(ir_LT(ref, ir_CONST_U32(0)));
ir_IF_TRUE(if_negative);
ir_MERGE_WITH_EMPTY_FALSE(if_negative);
ref = ir_PHI_2(IR_I32, ref, ir_CONST_I32(1));
ir_RETURN(ref);
ir_ref ref = ir_CALL_2(IR_ADDR, ir_CONST_FC_FUNC(handler), jit_FP(jit), jit_IP(jit));
zend_jit_vm_enter(jit, ref);
}
}
return 1;
@ -2014,10 +1982,8 @@ static int zend_jit_interrupt_handler_stub(zend_jit_ctx *jit)
{
ir_ref if_timeout, if_exception;
if (GCC_GLOBAL_REGS) {
// EX(opline) = opline
ir_STORE(jit_EX(opline), jit_IP(jit));
}
// EX(opline) = opline
ir_STORE(jit_EX(opline), jit_IP(jit));
ir_STORE(jit_EG(vm_interrupt), ir_CONST_U8(0));
if_timeout = ir_IF(ir_EQ(ir_LOAD_U8(jit_EG(timed_out)), ir_CONST_U8(0)));
@ -2039,7 +2005,7 @@ static int zend_jit_interrupt_handler_stub(zend_jit_ctx *jit)
if (GCC_GLOBAL_REGS) {
ir_TAILCALL(IR_VOID, ir_LOAD_A(jit_IP(jit)));
} else {
ir_RETURN(ir_CONST_I32(1));
zend_jit_vm_enter(jit, jit_IP(jit));
}
return 1;
}
@ -2059,7 +2025,7 @@ static int zend_jit_leave_function_handler_stub(zend_jit_ctx *jit)
} else if (GCC_GLOBAL_REGS) {
ir_TAILCALL_1(IR_VOID, ir_CONST_FC_FUNC(zend_jit_leave_nested_func_helper), call_info);
} else {
ir_TAILCALL_2(IR_I32, ir_CONST_FC_FUNC(zend_jit_leave_nested_func_helper), call_info, jit_FP(jit));
ir_TAILCALL_3(IR_ADDR, ir_CONST_FC_FUNC(zend_jit_leave_nested_func_helper), jit_FP(jit), jit_IP(jit), call_info);
}
ir_IF_TRUE(if_top);
@ -2070,7 +2036,7 @@ static int zend_jit_leave_function_handler_stub(zend_jit_ctx *jit)
} else if (GCC_GLOBAL_REGS) {
ir_TAILCALL_1(IR_VOID, ir_CONST_FC_FUNC(zend_jit_leave_top_func_helper), call_info);
} else {
ir_TAILCALL_2(IR_I32, ir_CONST_FC_FUNC(zend_jit_leave_top_func_helper), call_info, jit_FP(jit));
ir_TAILCALL_3(IR_ADDR, ir_CONST_FC_FUNC(zend_jit_leave_top_func_helper), jit_FP(jit), jit_IP(jit), call_info);
}
return 1;
@ -2209,9 +2175,7 @@ static int zend_jit_icall_throw_stub(zend_jit_ctx *jit)
// JIT: opline = EG(exception_op);
jit_STORE_IP(jit, jit_EG(exception_op));
if (GCC_GLOBAL_REGS) {
ir_STORE(jit_EX(opline), jit_IP(jit));
}
ir_STORE(jit_EX(opline), jit_IP(jit));
ir_IJMP(jit_STUB_ADDR(jit, jit_stub_exception_handler));
@ -2234,7 +2198,7 @@ static int zend_jit_leave_throw_stub(zend_jit_ctx *jit)
ir_MERGE_WITH_EMPTY_TRUE(if_set);
// JIT: opline = EG(exception_op);
jit_LOAD_IP(jit, jit_EG(exception_op));
jit_STORE_IP(jit, jit_EG(exception_op));
if (GCC_GLOBAL_REGS) {
ir_STORE(jit_EX(opline), jit_IP(jit));
@ -2242,7 +2206,7 @@ static int zend_jit_leave_throw_stub(zend_jit_ctx *jit)
// JIT: HANDLE_EXCEPTION()
ir_IJMP(jit_STUB_ADDR(jit, jit_stub_exception_handler));
} else {
ir_RETURN(ir_CONST_I32(2)); // ZEND_VM_LEAVE
zend_jit_vm_leave(jit, jit_IP(jit));
}
return 1;
@ -2339,13 +2303,7 @@ static int zend_jit_hybrid_loop_hot_counter_stub(zend_jit_ctx *jit)
static ir_ref _zend_jit_orig_opline_handler(zend_jit_ctx *jit, ir_ref offset)
{
ir_ref addr;
if (GCC_GLOBAL_REGS) {
addr = ir_ADD_A(offset, jit_IP(jit));
} else {
addr = ir_ADD_A(offset, ir_LOAD_A(jit_EX(opline)));
}
ir_ref addr = ir_ADD_A(offset, jit_IP(jit));
return ir_LOAD_A(addr);
}
@ -2433,7 +2391,7 @@ static int zend_jit_trace_halt_stub(zend_jit_ctx *jit)
jit_STORE_IP(jit, IR_NULL);
ir_RETURN(IR_VOID);
} else {
ir_RETURN(ir_CONST_I32(-1)); // ZEND_VM_RETURN
ir_RETURN(ir_CONST_ADDR(ZEND_VM_ENTER_BIT)); // ZEND_VM_RETURN
}
return 1;
}
@ -2443,7 +2401,7 @@ static int zend_jit_trace_escape_stub(zend_jit_ctx *jit)
if (GCC_GLOBAL_REGS) {
ir_TAILCALL(IR_VOID, ir_LOAD_A(jit_IP(jit)));
} else {
ir_RETURN(ir_CONST_I32(1)); // ZEND_VM_ENTER
zend_jit_vm_enter(jit, jit_IP(jit));
}
return 1;
@ -2453,10 +2411,8 @@ static int zend_jit_trace_exit_stub(zend_jit_ctx *jit)
{
ir_ref ref, ret, if_zero, addr;
if (GCC_GLOBAL_REGS) {
// EX(opline) = opline
ir_STORE(jit_EX(opline), jit_IP(jit));
}
// EX(opline) = opline
ir_STORE(jit_EX(opline), jit_IP(jit));
ret = ir_EXITCALL(ir_CONST_FC_FUNC(zend_jit_trace_exit));
@ -2464,14 +2420,15 @@ static int zend_jit_trace_exit_stub(zend_jit_ctx *jit)
ir_IF_TRUE(if_zero);
ref = jit_EG(current_execute_data);
jit_STORE_FP(jit, ir_LOAD_A(ref));
ref = ir_LOAD_A(jit_EX(opline));
jit_STORE_IP(jit, ref);
if (GCC_GLOBAL_REGS) {
ref = jit_EG(current_execute_data);
jit_STORE_FP(jit, ir_LOAD_A(ref));
ref = ir_LOAD_A(jit_EX(opline));
jit_STORE_IP(jit, ref);
ir_TAILCALL(IR_VOID, ir_LOAD_A(jit_IP(jit)));
} else {
ir_RETURN(ir_CONST_I32(1)); // ZEND_VM_ENTER
zend_jit_vm_enter(jit, ref);
}
ir_IF_FALSE(if_zero);
@ -2481,10 +2438,8 @@ static int zend_jit_trace_exit_stub(zend_jit_ctx *jit)
ref = jit_EG(current_execute_data);
jit_STORE_FP(jit, ir_LOAD_A(ref));
if (GCC_GLOBAL_REGS) {
ref = ir_LOAD_A(jit_EX(opline));
jit_STORE_IP(jit, ref);
}
ref = ir_LOAD_A(jit_EX(opline));
jit_STORE_IP(jit, ref);
// check for interrupt (try to avoid this ???)
zend_jit_check_timeout(jit, NULL, NULL);
@ -2496,9 +2451,8 @@ static int zend_jit_trace_exit_stub(zend_jit_ctx *jit)
#if defined(IR_TARGET_X86)
addr = ir_CAST_FC_FUNC(addr);
#endif
ref = ir_CALL_1(IR_I32, addr, jit_FP(jit));
ir_GUARD(ir_GE(ref, ir_CONST_I32(0)), jit_STUB_ADDR(jit, jit_stub_trace_halt));
ir_RETURN(ir_CONST_I32(1)); // ZEND_VM_ENTER
ref = ir_CALL_2(IR_ADDR, addr, jit_FP(jit), jit_IP(jit));
zend_jit_vm_enter(jit, ref);
}
return 1;
@ -2509,7 +2463,7 @@ static int zend_jit_undefined_offset_stub(zend_jit_ctx *jit)
if (GCC_GLOBAL_REGS) {
ir_TAILCALL(IR_VOID, ir_CONST_FC_FUNC(zend_jit_undefined_long_key));
} else {
ir_TAILCALL_1(IR_VOID, ir_CONST_FC_FUNC(zend_jit_undefined_long_key), jit_FP(jit));
ir_TAILCALL_2(IR_VOID, ir_CONST_FC_FUNC(zend_jit_undefined_long_key), jit_FP(jit), jit_IP(jit));
}
return 1;
@ -2520,7 +2474,7 @@ static int zend_jit_undefined_key_stub(zend_jit_ctx *jit)
if (GCC_GLOBAL_REGS) {
ir_TAILCALL(IR_VOID, ir_CONST_FC_FUNC(zend_jit_undefined_string_key));
} else {
ir_TAILCALL_1(IR_VOID, ir_CONST_FC_FUNC(zend_jit_undefined_string_key), jit_FP(jit));
ir_TAILCALL_2(IR_VOID, ir_CONST_FC_FUNC(zend_jit_undefined_string_key), jit_FP(jit), jit_IP(jit));
}
return 1;
@ -2702,6 +2656,10 @@ static void zend_jit_init_ctx(zend_jit_ctx *jit, uint32_t flags)
#else /* IR_TARGET_x86 */
jit->ctx.fixed_stack_frame_size = sizeof(void*) * 11; /* 4 saved registers and 7 spill slots (4 bytes) */
#endif
/* JIT-ed code is called only from execute_ex, which takes care
* of saving ZREG_FP, ZREG_IP when GCC_GLOBAL_REGS is 1, so we don't
* have to save them.
*/
if (GCC_GLOBAL_REGS) {
jit->ctx.fixed_save_regset = IR_REGSET_PRESERVED & ~((1<<ZREG_FP) | (1<<ZREG_IP));
} else {
@ -3153,8 +3111,10 @@ static void zend_jit_calc_trace_prologue_size(void)
zend_jit_init_ctx(jit, (zend_jit_vm_kind == ZEND_VM_KIND_CALL) ? 0 : IR_START_BR_TARGET);
if (!GCC_GLOBAL_REGS) {
ir_ref ref = ir_PARAM(IR_ADDR, "execute_data", 1);
jit_STORE_FP(jit, ref);
ir_ref execute_data_ref = ir_PARAM(IR_ADDR, "execute_data", 1);
ir_ref opline_ref = ir_PARAM(IR_ADDR, "opline", 2);
jit_STORE_FP(jit, execute_data_ref);
jit_STORE_IP(jit, opline_ref);
jit->ctx.flags |= IR_FASTCALL_FUNC;
}
@ -3960,18 +3920,10 @@ static int jit_CMP_IP(zend_jit_ctx *jit, ir_op op, const zend_op *next_opline)
ir_ref ref;
#if 1
if (GCC_GLOBAL_REGS) {
ref = jit_IP32(jit);
} else {
ref = ir_LOAD_U32(jit_EX(opline));
}
ref = jit_IP32(jit);
ref = ir_CMP_OP(op, ref, ir_CONST_U32((uint32_t)(uintptr_t)next_opline));
#else
if (GCC_GLOBAL_REGS) {
ref = jit_IP(jit);
} else {
ref = ir_LOAD_A(jit_EX(opline));
}
ref = jit_IP(jit);
ref = ir_CMP_OP(op, ref, ir_CONST_ADDR(next_opline));
#endif
return ref;
@ -4144,10 +4096,13 @@ static void zend_jit_recv_entry(zend_jit_ctx *jit, int b)
/* Insert a MERGE block with additional ENTRY input between predecessor and this one */
ir_ENTRY(ref, bb->start);
if (!GCC_GLOBAL_REGS) {
/* 2 is hardcoded reference to IR_PARAM */
/* 2 and 3 are hardcoded reference to IR_PARAMs */
ZEND_ASSERT(jit->ctx.ir_base[2].op == IR_PARAM);
ZEND_ASSERT(jit->ctx.ir_base[2].op3 == 1);
jit_STORE_FP(jit, 2);
ZEND_ASSERT(jit->ctx.ir_base[3].op == IR_PARAM);
ZEND_ASSERT(jit->ctx.ir_base[3].op3 == 2);
jit_STORE_IP(jit, 3);
}
ir_MERGE_WITH(ref);
@ -4162,10 +4117,13 @@ static void zend_jit_osr_entry(zend_jit_ctx *jit, int b)
/* Insert a MERGE block with additional ENTRY input between predecessor and this one */
ir_ENTRY(ref, bb->start);
if (!GCC_GLOBAL_REGS) {
/* 2 is hardcoded reference to IR_PARAM */
/* 2 and 3 are hardcoded reference to IR_PARAMs */
ZEND_ASSERT(jit->ctx.ir_base[2].op == IR_PARAM);
ZEND_ASSERT(jit->ctx.ir_base[2].op3 == 1);
jit_STORE_FP(jit, 2);
ZEND_ASSERT(jit->ctx.ir_base[3].op == IR_PARAM);
ZEND_ASSERT(jit->ctx.ir_base[3].op3 == 2);
jit_STORE_IP(jit, 3);
}
ir_MERGE_WITH(ref);
@ -4175,17 +4133,19 @@ static ir_ref zend_jit_continue_entry(zend_jit_ctx *jit, ir_ref src, unsigned in
{
ir_ENTRY(src, label);
if (!GCC_GLOBAL_REGS) {
/* 2 is hardcoded reference to IR_PARAM */
/* 2 and 3 are hardcoded reference to IR_PARAMs */
ZEND_ASSERT(jit->ctx.ir_base[2].op == IR_PARAM);
ZEND_ASSERT(jit->ctx.ir_base[2].op3 == 1);
jit_STORE_FP(jit, 2);
ZEND_ASSERT(jit->ctx.ir_base[3].op == IR_PARAM);
ZEND_ASSERT(jit->ctx.ir_base[3].op3 == 2);
jit_STORE_IP(jit, 3);
}
return ir_END();
}
static int zend_jit_handler(zend_jit_ctx *jit, const zend_op *opline, int may_throw)
{
ir_ref ref;
const void *handler;
zend_jit_set_ip(jit, opline);
@ -4197,8 +4157,8 @@ static int zend_jit_handler(zend_jit_ctx *jit, const zend_op *opline, int may_th
if (GCC_GLOBAL_REGS) {
ir_CALL(IR_VOID, ir_CONST_FUNC(handler));
} else {
ref = jit_FP(jit);
ref = ir_CALL_1(IR_VOID, ir_CONST_FC_FUNC(handler), ref);
ir_ref ip = ir_CALL_2(IR_ADDR, ir_CONST_FC_FUNC(handler), jit_FP(jit), jit_IP(jit));
jit_STORE_IP(jit, ip);
}
if (may_throw) {
zend_jit_check_exception(jit);
@ -4256,12 +4216,10 @@ static int zend_jit_tail_handler(zend_jit_ctx *jit, const zend_op *opline)
|| opline->opcode == ZEND_MATCH_ERROR
|| opline->opcode == ZEND_THROW
|| opline->opcode == ZEND_VERIFY_NEVER_TYPE)) {
ref = jit_FP(jit);
ir_CALL_1(IR_I32, ir_CONST_FC_FUNC(handler), ref);
ir_RETURN(ir_CONST_I32(1));
ir_ref ip = ir_CALL_2(IR_ADDR, ir_CONST_FC_FUNC(handler), jit_FP(jit), jit_IP(jit));
zend_jit_vm_enter(jit, ip);
} else {
ref = jit_FP(jit);
ir_TAILCALL_1(IR_I32, ir_CONST_FC_FUNC(handler), ref);
ir_TAILCALL_2(IR_ADDR, ir_CONST_FC_FUNC(handler), jit_FP(jit), jit_IP(jit));
}
}
if (jit->b >= 0) {
@ -10319,7 +10277,7 @@ static int zend_jit_do_fcall(zend_jit_ctx *jit, const zend_op *opline, const zen
if (num_args) {
ip = ir_ADD_OFFSET(ip, num_args * sizeof(zend_op));
}
jit_LOAD_IP(jit, ip);
jit_STORE_IP(jit, ip);
}
if (!trace && op_array == &func->op_array && call_num_args >= op_array->required_num_args) {
@ -10341,7 +10299,7 @@ static int zend_jit_do_fcall(zend_jit_ctx *jit, const zend_op *opline, const zen
}
ip = ir_LOAD_A(ir_ADD_OFFSET(func_ref, offsetof(zend_op_array, opcodes)));
}
jit_LOAD_IP(jit, ip);
jit_STORE_IP(jit, ip);
helper = ir_CONST_FC_FUNC(zend_jit_copy_extra_args_helper);
} else {
helper = ir_CONST_FC_FUNC(zend_jit_copy_extra_args_helper_no_skip_recv);
@ -10349,7 +10307,8 @@ static int zend_jit_do_fcall(zend_jit_ctx *jit, const zend_op *opline, const zen
if (GCC_GLOBAL_REGS) {
ir_CALL(IR_VOID, helper);
} else {
ir_CALL_1(IR_VOID, helper, jit_FP(jit));
ir_ref ref = ir_CALL_2(IR_ADDR, helper, jit_FP(jit), jit_IP(jit));
jit_STORE_IP(jit, ref);
}
}
} else {
@ -10365,7 +10324,7 @@ static int zend_jit_do_fcall(zend_jit_ctx *jit, const zend_op *opline, const zen
}
ip = ir_LOAD_A(ir_ADD_OFFSET(func_ref, offsetof(zend_op_array, opcodes)));
}
jit_LOAD_IP(jit, ip);
jit_STORE_IP(jit, ip);
// JIT: num_args = EX_NUM_ARGS();
ir_ref num_args, first_extra_arg;
@ -10385,7 +10344,8 @@ static int zend_jit_do_fcall(zend_jit_ctx *jit, const zend_op *opline, const zen
if (GCC_GLOBAL_REGS) {
ir_CALL(IR_VOID, ir_CONST_FC_FUNC(zend_jit_copy_extra_args_helper));
} else {
ir_CALL_1(IR_VOID, ir_CONST_FC_FUNC(zend_jit_copy_extra_args_helper), jit_FP(jit));
ir_ref ref = ir_CALL_2(IR_ADDR, ir_CONST_FC_FUNC(zend_jit_copy_extra_args_helper), jit_FP(jit), jit_IP(jit));
jit_STORE_IP(jit, ref);
}
ir_END_list(merge_inputs);
ir_IF_FALSE(if_extra_args);
@ -10407,13 +10367,7 @@ static int zend_jit_do_fcall(zend_jit_ctx *jit, const zend_op *opline, const zen
ref = ir_ZEXT_A(ref);
}
if (GCC_GLOBAL_REGS) {
jit_STORE_IP(jit, ir_ADD_A(jit_IP(jit), ref));
} else {
ir_ref addr = jit_EX(opline);
ir_STORE(addr, ir_ADD_A(ir_LOAD_A(addr), ref));
}
jit_STORE_IP(jit, ir_ADD_A(jit_IP(jit), ref));
}
ir_END_list(merge_inputs);
@ -10462,7 +10416,7 @@ static int zend_jit_do_fcall(zend_jit_ctx *jit, const zend_op *opline, const zen
if (trace && (trace->op != ZEND_JIT_TRACE_END || trace->stop < ZEND_JIT_TRACE_STOP_INTERPRETER)) {
ZEND_ASSERT(trace[1].op == ZEND_JIT_TRACE_VM || trace[1].op == ZEND_JIT_TRACE_END);
jit_SET_EX_OPLINE(jit, trace[1].opline);
} else if (GCC_GLOBAL_REGS) {
} else {
// EX(opline) = opline
ir_STORE(jit_EX(opline), jit_IP(jit));
}
@ -10529,7 +10483,7 @@ static int zend_jit_do_fcall(zend_jit_ctx *jit, const zend_op *opline, const zen
if (GCC_GLOBAL_REGS) {
ir_TAILCALL(IR_VOID, ir_LOAD_A(jit_IP(jit)));
} else {
ir_RETURN(ir_CONST_I32(1));
zend_jit_vm_enter(jit, jit_IP(jit));
}
} while (0);
@ -11129,7 +11083,7 @@ static int zend_jit_leave_func(zend_jit_ctx *jit,
ir_IF_TRUE_cold(if_slow);
if (!GCC_GLOBAL_REGS) {
ref = ir_CALL_1(IR_I32, ir_CONST_FC_FUNC(zend_jit_leave_func_helper), jit_FP(jit));
ref = ir_CALL_2(IR_ADDR, ir_CONST_FC_FUNC(zend_jit_leave_func_helper), jit_FP(jit), jit_IP(jit));
} else {
ir_CALL(IR_VOID, ir_CONST_FC_FUNC(zend_jit_leave_func_helper));
}
@ -11145,7 +11099,8 @@ static int zend_jit_leave_func(zend_jit_ctx *jit,
} else if (GCC_GLOBAL_REGS) {
ir_GUARD(jit_IP(jit), jit_STUB_ADDR(jit, jit_stub_trace_halt));
} else {
ir_GUARD(ir_GE(ref, ir_CONST_I32(0)), jit_STUB_ADDR(jit, jit_stub_trace_halt));
ir_GUARD(ir_NE(ref, ir_CONST_ADDR(ZEND_VM_ENTER_BIT)), jit_STUB_ADDR(jit, jit_stub_trace_halt));
jit_STORE_IP(jit, ref);
}
}
@ -11213,26 +11168,20 @@ static int zend_jit_leave_func(zend_jit_ctx *jit,
&& (JIT_G(current_frame) && !TRACE_FRAME_IS_UNKNOWN_RETURN(JIT_G(current_frame)))) {
zend_jit_reset_last_valid_opline(jit);
} else {
if (GCC_GLOBAL_REGS) {
/* We add extra RLOAD and RSTORE to make fusion for persistent register
* mov (%FP), %IP
* add $0x1c, %IP
* The naive (commented) code leads to extra register allocation and move.
* mov (%FP), %tmp
* add $0x1c, %tmp
* mov %tmp, %FP
*/
/* We add extra RLOAD and RSTORE to make fusion for persistent register
* mov (%FP), %IP
* add $0x1c, %IP
* The naive (commented) code leads to extra register allocation and move.
* mov (%FP), %tmp
* add $0x1c, %tmp
* mov %tmp, %FP
*/
#if 0
jit_STORE_IP(jit, ir_ADD_OFFSET(ir_LOAD_A(jit_EX(opline)), sizeof(zend_op)));
jit_STORE_IP(jit, ir_ADD_OFFSET(ir_LOAD_A(jit_EX(opline)), sizeof(zend_op)));
#else
jit_STORE_IP(jit, ir_LOAD_A(jit_EX(opline)));
jit_STORE_IP(jit, ir_ADD_OFFSET(jit_IP(jit), sizeof(zend_op)));
jit_STORE_IP(jit, ir_LOAD_A(jit_EX(opline)));
jit_STORE_IP(jit, ir_ADD_OFFSET(jit_IP(jit), sizeof(zend_op)));
#endif
} else {
ir_ref ref = jit_EX(opline);
ir_STORE(ref, ir_ADD_OFFSET(ir_LOAD_A(ref), sizeof(zend_op)));
}
}
if (cold_path) {
@ -11295,20 +11244,14 @@ static int zend_jit_leave_func(zend_jit_ctx *jit,
// JIT: if (EG(exception))
ir_GUARD_NOT(ir_LOAD_A(jit_EG(exception)), jit_STUB_ADDR(jit, jit_stub_leave_throw));
// JIT: opline = EX(opline) + 1
if (GCC_GLOBAL_REGS) {
jit_STORE_IP(jit, ir_LOAD_A(jit_EX(opline)));
jit_STORE_IP(jit, ir_ADD_OFFSET(jit_IP(jit), sizeof(zend_op)));
} else {
ir_ref ref = jit_EX(opline);
ir_STORE(ref, ir_ADD_OFFSET(ir_LOAD_A(ref), sizeof(zend_op)));
}
jit_STORE_IP(jit, ir_LOAD_A(jit_EX(opline)));
jit_STORE_IP(jit, ir_ADD_OFFSET(jit_IP(jit), sizeof(zend_op)));
}
if (GCC_GLOBAL_REGS) {
ir_TAILCALL(IR_VOID, ir_LOAD_A(jit_IP(jit)));
} else {
ir_RETURN(ir_CONST_I32(2)); // ZEND_VM_LEAVE
zend_jit_vm_leave(jit, jit_IP(jit));
}
jit->b = -1;
@ -16745,8 +16688,10 @@ static int zend_jit_start(zend_jit_ctx *jit, const zend_op_array *op_array, zend
jit->bb_edges = zend_arena_calloc(&CG(arena), count, sizeof(ir_ref));
if (!GCC_GLOBAL_REGS) {
ir_ref ref = ir_PARAM(IR_ADDR, "execute_data", 1);
jit_STORE_FP(jit, ref);
ir_ref execute_data_ref = ir_PARAM(IR_ADDR, "execute_data", 1);
ir_ref opline_ref = ir_PARAM(IR_ADDR, "opline", 2);
jit_STORE_FP(jit, execute_data_ref);
jit_STORE_IP(jit, opline_ref);
jit->ctx.flags |= IR_FASTCALL_FUNC;
}
@ -17129,8 +17074,18 @@ static int zend_jit_trace_handler(zend_jit_ctx *jit, const zend_op_array *op_arr
if (GCC_GLOBAL_REGS) {
ir_CALL(IR_VOID, ir_CONST_FUNC(handler));
} else {
ref = jit_FP(jit);
ref = ir_CALL_1(IR_I32, ir_CONST_FC_FUNC(handler), ref);
ref = ir_CALL_2(IR_ADDR, ir_CONST_FC_FUNC(handler), jit_FP(jit), jit_IP(jit));
if (opline->opcode == ZEND_RETURN ||
opline->opcode == ZEND_RETURN_BY_REF ||
opline->opcode == ZEND_DO_UCALL ||
opline->opcode == ZEND_DO_FCALL_BY_NAME ||
opline->opcode == ZEND_DO_FCALL ||
opline->opcode == ZEND_GENERATOR_CREATE) {
jit_STORE_IP(jit, ir_AND_A(ref, ir_CONST_ADDR(~ZEND_VM_ENTER_BIT)));
} else {
jit_STORE_IP(jit, ref);
}
}
if (may_throw
&& opline->opcode != ZEND_RETURN
@ -17170,10 +17125,9 @@ static int zend_jit_trace_handler(zend_jit_ctx *jit, const zend_op_array *op_arr
ir_GUARD(ir_NE(jit_IP(jit), ir_CONST_ADDR(zend_jit_halt_op)),
jit_STUB_ADDR(jit, jit_stub_trace_halt));
}
} else if (GCC_GLOBAL_REGS) {
ir_GUARD(jit_IP(jit), jit_STUB_ADDR(jit, jit_stub_trace_halt));
} else {
ir_GUARD(ir_GE(ref, ir_CONST_I32(0)), jit_STUB_ADDR(jit, jit_stub_trace_halt));
/* IP has been cleared of ZEND_VM_ENTER_BIT already */
ir_GUARD(jit_IP(jit), jit_STUB_ADDR(jit, jit_stub_trace_halt));
}
} else if (opline->opcode == ZEND_GENERATOR_RETURN ||
opline->opcode == ZEND_YIELD ||
@ -17309,8 +17263,10 @@ static int zend_jit_trace_start(zend_jit_ctx *jit,
if (!GCC_GLOBAL_REGS) {
if (!parent) {
ir_ref ref = ir_PARAM(IR_ADDR, "execute_data", 1);
jit_STORE_FP(jit, ref);
ir_ref execute_data_ref = ir_PARAM(IR_ADDR, "execute_data", 1);
ir_ref opline_ref = ir_PARAM(IR_ADDR, "opline", 2);
jit_STORE_FP(jit, execute_data_ref);
jit_STORE_IP(jit, opline_ref);
jit->ctx.flags |= IR_FASTCALL_FUNC;
}
}
@ -17423,7 +17379,7 @@ static int zend_jit_trace_return(zend_jit_ctx *jit, bool original_handler, const
#if defined(IR_TARGET_X86)
addr = ir_CAST_FC_FUNC(addr);
#endif
ref = ir_CALL_1(IR_I32, addr, jit_FP(jit));
ref = ir_CALL_2(IR_ADDR, addr, jit_FP(jit), jit_IP(jit));
if (opline &&
(opline->opcode == ZEND_RETURN
|| opline->opcode == ZEND_RETURN_BY_REF
@ -17431,11 +17387,11 @@ static int zend_jit_trace_return(zend_jit_ctx *jit, bool original_handler, const
|| opline->opcode == ZEND_GENERATOR_CREATE
|| opline->opcode == ZEND_YIELD
|| opline->opcode == ZEND_YIELD_FROM)) {
ir_RETURN(ref);
zend_jit_vm_enter(jit, ref);
return 1;
}
}
ir_RETURN(ir_CONST_I32(2)); // ZEND_VM_LEAVE
zend_jit_vm_enter(jit, jit_IP(jit));
}
return 1;
}

7
ext/opcache/jit/zend_jit_trace.c
View File

@ -16,6 +16,11 @@
+----------------------------------------------------------------------+
*/
#include "zend_jit.h"
#include "zend_jit_internal.h"
#include "zend_shared_alloc.h"
#include "ir/ir.h"
static zend_jit_trace_info *zend_jit_traces = NULL;
static const void **zend_jit_exit_groups = NULL;
@ -7433,7 +7438,7 @@ static const void *zend_jit_trace_exit_to_vm(uint32_t trace_num, uint32_t exit_n
original_handler = 1;
}
}
zend_jit_set_ip_ex(&ctx, opline, original_handler);
zend_jit_set_ip(&ctx, opline);
}
zend_jit_trace_return(&ctx, original_handler, opline);

99
ext/opcache/jit/zend_jit_vm_helpers.c
View File

@ -46,7 +46,7 @@ register const zend_op* volatile opline __asm__("x28");
# pragma GCC diagnostic warning "-Wvolatile-register-var"
#endif
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_nested_func_helper(uint32_t call_info EXECUTE_DATA_DC)
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_nested_func_helper(ZEND_OPCODE_HANDLER_ARGS_EX uint32_t call_info)
{
zend_execute_data *old_execute_data;
@ -75,19 +75,19 @@ ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_nested_func_helper(uint32_t
zval_ptr_dtor(EX_VAR(old_opline->result.var));
}
#ifndef HAVE_GCC_GLOBAL_REGS
return 2; // ZEND_VM_LEAVE
return (zend_op*)((uintptr_t)EG(current_execute_data)->opline | ZEND_VM_ENTER_BIT);
#endif
} else {
EX(opline)++;
#ifdef HAVE_GCC_GLOBAL_REGS
opline = EX(opline);
#else
return 2; // ZEND_VM_LEAVE
return (zend_op*)((uintptr_t)EX(opline) | ZEND_VM_ENTER_BIT);
#endif
}
}
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_top_func_helper(uint32_t call_info EXECUTE_DATA_DC)
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_top_func_helper(ZEND_OPCODE_HANDLER_ARGS_EX uint32_t call_info)
{
if (UNEXPECTED(call_info & (ZEND_CALL_HAS_SYMBOL_TABLE|ZEND_CALL_FREE_EXTRA_ARGS))) {
if (UNEXPECTED(call_info & ZEND_CALL_HAS_SYMBOL_TABLE)) {
@ -105,11 +105,11 @@ ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_top_func_helper(uint32_t ca
#ifdef HAVE_GCC_GLOBAL_REGS
opline = zend_jit_halt_op;
#else
return -1; // ZEND_VM_RETURN
return (const zend_op*)ZEND_VM_ENTER_BIT; // ZEND_VM_RETURN
#endif
}
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_func_helper(EXECUTE_DATA_D)
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_func_helper(ZEND_OPCODE_HANDLER_ARGS)
{
uint32_t call_info = EX_CALL_INFO();
@ -120,7 +120,7 @@ ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_leave_func_helper(EXECUTE_DATA_D)
}
}
static void ZEND_FASTCALL zend_jit_copy_extra_args_helper_ex(bool skip_recv EXECUTE_DATA_DC)
static ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_copy_extra_args_helper_ex(ZEND_OPCODE_HANDLER_ARGS_EX bool skip_recv)
{
zend_op_array *op_array = &EX(func)->op_array;
@ -132,11 +132,7 @@ static void ZEND_FASTCALL zend_jit_copy_extra_args_helper_ex(bool skip_recv EXEC
if (skip_recv && EXPECTED((op_array->fn_flags & ZEND_ACC_HAS_TYPE_HINTS) == 0)) {
/* Skip useless ZEND_RECV and ZEND_RECV_INIT opcodes */
#ifdef HAVE_GCC_GLOBAL_REGS
opline += first_extra_arg;
#else
EX(opline) += first_extra_arg;
#endif
}
/* move extra args into separate array after all CV and TMP vars */
@ -164,16 +160,20 @@ static void ZEND_FASTCALL zend_jit_copy_extra_args_helper_ex(bool skip_recv EXEC
} while (src != end);
}
}
#ifndef HAVE_GCC_GLOBAL_REGS
return opline;
#endif
}
void ZEND_FASTCALL zend_jit_copy_extra_args_helper(EXECUTE_DATA_D)
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_copy_extra_args_helper(ZEND_OPCODE_HANDLER_ARGS)
{
zend_jit_copy_extra_args_helper_ex(true EXECUTE_DATA_CC);
return zend_jit_copy_extra_args_helper_ex(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX true);
}
void ZEND_FASTCALL zend_jit_copy_extra_args_helper_no_skip_recv(EXECUTE_DATA_D)
ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_copy_extra_args_helper_no_skip_recv(ZEND_OPCODE_HANDLER_ARGS)
{
zend_jit_copy_extra_args_helper_ex(false EXECUTE_DATA_CC);
return zend_jit_copy_extra_args_helper_ex(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU_EX false);
}
bool ZEND_FASTCALL zend_jit_deprecated_helper(OPLINE_D)
@ -312,9 +312,6 @@ ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_func_counter_helper(ZEND_OPCODE_H
{
zend_jit_op_array_hot_extension *jit_extension =
(zend_jit_op_array_hot_extension*)ZEND_FUNC_INFO(&EX(func)->op_array);
#ifndef HAVE_GCC_GLOBAL_REGS
const zend_op *opline = EX(opline);
#endif
*(jit_extension->counter) -= ((ZEND_JIT_COUNTER_INIT + JIT_G(hot_func) - 1) / JIT_G(hot_func));
@ -332,9 +329,6 @@ ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_loop_counter_helper(ZEND_OPCODE_H
{
zend_jit_op_array_hot_extension *jit_extension =
(zend_jit_op_array_hot_extension*)ZEND_FUNC_INFO(&EX(func)->op_array);
#ifndef HAVE_GCC_GLOBAL_REGS
const zend_op *opline = EX(opline);
#endif
*(jit_extension->counter) -= ((ZEND_JIT_COUNTER_INIT + JIT_G(hot_loop) - 1) / JIT_G(hot_loop));
@ -403,14 +397,11 @@ zend_constant* ZEND_FASTCALL zend_jit_check_constant(const zval *key)
return _zend_quick_get_constant(key, 0, 1);
}
static zend_always_inline ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_trace_counter_helper(uint32_t cost ZEND_OPCODE_HANDLER_ARGS_DC)
static zend_always_inline ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_trace_counter_helper(ZEND_OPCODE_HANDLER_ARGS_EX uint32_t cost)
{
zend_jit_op_array_trace_extension *jit_extension =
(zend_jit_op_array_trace_extension*)ZEND_FUNC_INFO(&EX(func)->op_array);
size_t offset = jit_extension->offset;
#ifndef HAVE_GCC_GLOBAL_REGS
const zend_op *opline = EX(opline);
#endif
*(ZEND_OP_TRACE_INFO(opline, offset)->counter) -= cost;
@ -421,15 +412,15 @@ static zend_always_inline ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_trace_c
opline = NULL;
return;
#else
return -1;
return (const zend_op*)ZEND_VM_ENTER_BIT; // ZEND_VM_RETURN()
#endif
}
#ifdef HAVE_GCC_GLOBAL_REGS
execute_data = EG(current_execute_data);
opline = execute_data ? EX(opline) : NULL;
#ifdef HAVE_GCC_GLOBAL_REGS
return;
#else
return 1;
return (const zend_op*)((uintptr_t)opline | ZEND_VM_ENTER_BIT); // ZEND_VM_ENTER() / ZEND_VM_RETURN()
#endif
} else {
zend_vm_opcode_handler_t handler = (zend_vm_opcode_handler_t)ZEND_OP_TRACE_INFO(opline, offset)->orig_handler;
@ -678,15 +669,14 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data *ex,
int last_loop_level = -1;
const zend_op *last_loop_opline = NULL;
const zend_op_array *unrolled_calls[ZEND_JIT_TRACE_MAX_CALL_DEPTH + ZEND_JIT_TRACE_MAX_RET_DEPTH];
#ifdef HAVE_GCC_GLOBAL_REGS
zend_execute_data *prev_execute_data = ex;
#ifdef HAVE_GCC_GLOBAL_REGS
execute_data = ex;
opline = EX(opline) = op;
#else
int rc;
zend_execute_data *execute_data = ex;
const zend_op *opline = EX(opline);
const zend_op *opline = EX(opline) = op;
#endif
zend_execute_data *prev_call = EX(call);
@ -992,40 +982,29 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data *ex,
#ifdef HAVE_GCC_GLOBAL_REGS
handler();
if (UNEXPECTED(opline == zend_jit_halt_op)) {
#else
opline = handler(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU);
if (UNEXPECTED(((uintptr_t)opline & ~ZEND_VM_ENTER_BIT) == 0)) {
#endif
stop = ZEND_JIT_TRACE_STOP_RETURN;
opline = NULL;
halt = ZEND_JIT_TRACE_HALT;
break;
}
if (UNEXPECTED(execute_data != prev_execute_data)) {
#else
rc = handler(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU);
if (rc != 0) {
if (rc < 0) {
stop = ZEND_JIT_TRACE_STOP_RETURN;
opline = NULL;
halt = ZEND_JIT_TRACE_HALT;
break;
} else if (execute_data == EG(current_execute_data)) {
/* return after interrupt handler */
rc = 0;
}
#ifndef HAVE_GCC_GLOBAL_REGS
if ((uintptr_t)opline & ZEND_VM_ENTER_BIT) {
opline = (const zend_op*)((uintptr_t)opline & ~ZEND_VM_ENTER_BIT);
execute_data = EG(current_execute_data);
opline = EX(opline);
}
#endif
if (UNEXPECTED(execute_data != prev_execute_data)) {
op_array = &EX(func)->op_array;
jit_extension =
(zend_jit_op_array_trace_extension*)ZEND_FUNC_INFO(op_array);
if (UNEXPECTED(!jit_extension)
|| UNEXPECTED(!(jit_extension->func_info.flags & ZEND_FUNC_JIT_ON_HOT_TRACE))) {
#ifdef HAVE_GCC_GLOBAL_REGS
if (execute_data->prev_execute_data != prev_execute_data) {
#else
if (rc < 0) {
#endif
stop = ZEND_JIT_TRACE_STOP_RETURN;
} else {
stop = ZEND_JIT_TRACE_STOP_INTERPRETER;
}
break;
@ -1036,13 +1015,7 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data *ex,
op_array = jit_extension->op_array;
}
#ifdef HAVE_GCC_GLOBAL_REGS
if (execute_data->prev_execute_data == prev_execute_data) {
#else
if (rc == 0) {
/* pass */
} else if (rc == 1) {
#endif
/* Enter into function */
prev_call = NULL;
if (level > ZEND_JIT_TRACE_MAX_CALL_DEPTH) {
@ -1164,9 +1137,7 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data *ex,
TRACE_RECORD(ZEND_JIT_TRACE_BACK, 0, op_array);
}
}
#ifdef HAVE_GCC_GLOBAL_REGS
prev_execute_data = execute_data;
#endif
}
if (EX(call) != prev_call) {
if (EX(call)
@ -1191,10 +1162,6 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data *ex,
}
}
#ifndef HAVE_GCC_GLOBAL_REGS
opline = EX(opline);
#endif
if (!func
|| (func->common.fn_flags & ZEND_ACC_CALL_VIA_TRAMPOLINE)
|| (func->common.fn_flags & ZEND_ACC_NEVER_CACHE)
@ -1219,10 +1186,6 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data *ex,
prev_call = EX(call);
}
#ifndef HAVE_GCC_GLOBAL_REGS
opline = EX(opline);
#endif
if (UNEXPECTED(opline->opcode == ZEND_HANDLE_EXCEPTION)) {
/* Abort trace because of exception */
stop = ZEND_JIT_TRACE_STOP_EXCEPTION;
@ -1346,11 +1309,9 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data *ex,
TRACE_END(ZEND_JIT_TRACE_END, stop, end_opline);
#ifdef HAVE_GCC_GLOBAL_REGS
if (!halt) {
EX(opline) = opline;
}
#endif
#ifdef HAVE_GCC_GLOBAL_REGS
execute_data = save_execute_data;

4
sapi/phpdbg/tests/watch_006.phpt
View File

@ -2,9 +2,7 @@
Test multiple watch elements pointing to the same watchpoint
--SKIPIF--
<?php
if (PHP_INT_SIZE == 4) {
die("xfail There may be flaws in the implementation of watchpoints that cause failures");
}
die("xfail There may be flaws in the implementation of watchpoints that cause failures");
if (getenv('SKIP_ASAN')) {
die("skip intentionally causes segfaults");
}

Write Preview
Loading…
Cancel
Save