Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 895 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4963. Min Act: -0.1700

Data Index: 8451451 (Open Web Text)

Max Activating Token Index: 481

Text #1

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.5224. Min Act: -0.1700

Data Index: 8549196 (Open Web Text)

Max Activating Token Index: 659

Text #2

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.5261. Min Act: -0.1700

Data Index: 1699153 (Open Web Text)

Max Activating Token Index: 795

Text #3

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4925. Min Act: -0.1700

Data Index: 2760418 (Open Web Text)

Max Activating Token Index: 482

Text #4

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4921. Min Act: -0.1700

Data Index: 8411811 (Open Web Text)

Max Activating Token Index: 628

Text #5

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4849. Min Act: -0.1700

Data Index: 522634 (Open Web Text)

Max Activating Token Index: 399

Text #6

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4479. Min Act: -0.1700

Data Index: 716176 (Open Web Text)

Max Activating Token Index: 111

Text #7

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4477. Min Act: -0.1700

Data Index: 4776488 (Open Web Text)

Max Activating Token Index: 539

Text #8

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4492. Min Act: -0.1700

Data Index: 7284166 (Open Web Text)

Max Activating Token Index: 569

Text #9

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4178. Min Act: -0.1700

Data Index: 3273593 (Open Web Text)

Max Activating Token Index: 796

Text #10

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4594. Min Act: -0.1700

Data Index: 1363036 (Open Web Text)

Max Activating Token Index: 604

Text #11

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4035. Min Act: -0.1700

Data Index: 282030 (Open Web Text)

Max Activating Token Index: 582

Text #12

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4127. Min Act: -0.1700

Data Index: 8795575 (Open Web Text)

Max Activating Token Index: 576

Text #13

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4264. Min Act: -0.1700

Data Index: 7551966 (Open Web Text)

Max Activating Token Index: 1010

Text #14

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4131. Min Act: -0.1700

Data Index: 7384255 (Open Web Text)

Max Activating Token Index: 450

Text #15

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4379. Min Act: -0.1700

Data Index: 6867540 (Open Web Text)

Max Activating Token Index: 935

Text #16

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.3681. Min Act: -0.1700

Data Index: 7312323 (Open Web Text)

Max Activating Token Index: 788

Text #17

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4265. Min Act: -0.1700

Data Index: 5730051 (Open Web Text)

Max Activating Token Index: 1008

Text #18

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.4156. Min Act: -0.1700

Data Index: 5303540 (Open Web Text)

Max Activating Token Index: 486

Text #19

Max Range: 3.5261. Min Range: -3.5261

Max Act: 3.3869. Min Act: -0.1700

Data Index: 1435418 (Open Web Text)

Max Activating Token Index: 987