Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2674 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5898. Min Act: -0.1700

Data Index: 6472745 (Open Web Text)

Max Activating Token Index: 80

Text #1

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5648. Min Act: -0.1700

Data Index: 7004038 (Open Web Text)

Max Activating Token Index: 774

Text #2

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4984. Min Act: -0.1700

Data Index: 2967409 (Open Web Text)

Max Activating Token Index: 567

Text #3

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4973. Min Act: -0.1700

Data Index: 8176078 (Open Web Text)

Max Activating Token Index: 916

Text #4

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4904. Min Act: -0.1700

Data Index: 3718440 (Open Web Text)

Max Activating Token Index: 612

Text #5

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4651. Min Act: -0.1700

Data Index: 6778733 (Open Web Text)

Max Activating Token Index: 657

Text #6

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4426. Min Act: -0.1700

Data Index: 5906600 (Open Web Text)

Max Activating Token Index: 492

Text #7

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4170. Min Act: -0.1700

Data Index: 516914 (Open Web Text)

Max Activating Token Index: 975

Text #8

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3958. Min Act: -0.1700

Data Index: 6284909 (Open Web Text)

Max Activating Token Index: 842

Text #9

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3412. Min Act: -0.1700

Data Index: 2676693 (Open Web Text)

Max Activating Token Index: 844

Text #10

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3817. Min Act: -0.1700

Data Index: 2588572 (Open Web Text)

Max Activating Token Index: 282

Text #11

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3537. Min Act: -0.1700

Data Index: 2636502 (Open Web Text)

Max Activating Token Index: 620

Text #12

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3984. Min Act: -0.1700

Data Index: 6352264 (Open Web Text)

Max Activating Token Index: 966

Text #13

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.2366. Min Act: -0.1700

Data Index: 6260854 (Open Web Text)

Max Activating Token Index: 423

Text #14

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3287. Min Act: -0.1700

Data Index: 7527557 (Open Web Text)

Max Activating Token Index: 776

Text #15

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.2856. Min Act: -0.1700

Data Index: 8439614 (Open Web Text)

Max Activating Token Index: 275

Text #16

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3009. Min Act: -0.1700

Data Index: 4577093 (Open Web Text)

Max Activating Token Index: 265

Text #17

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3036. Min Act: -0.1700

Data Index: 6506165 (Open Web Text)

Max Activating Token Index: 578

Text #18

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3319. Min Act: -0.1700

Data Index: 751103 (Open Web Text)

Max Activating Token Index: 638

Text #19

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.3129. Min Act: -0.1700

Data Index: 4944484 (Open Web Text)

Max Activating Token Index: 363