Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3047 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.3728. Min Act: -0.1700

Data Index: 3208100 (Open Web Text)

Max Activating Token Index: 857

Text #1

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.3651. Min Act: -0.1700

Data Index: 730430 (Open Web Text)

Max Activating Token Index: 618

Text #2

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.3228. Min Act: -0.1700

Data Index: 5650589 (Open Web Text)

Max Activating Token Index: 176

Text #3

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2879. Min Act: -0.1700

Data Index: 350355 (Open Web Text)

Max Activating Token Index: 657

Text #4

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2383. Min Act: -0.1700

Data Index: 5412625 (Open Web Text)

Max Activating Token Index: 423

Text #5

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2260. Min Act: -0.1700

Data Index: 3444910 (Open Web Text)

Max Activating Token Index: 975

Text #6

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2471. Min Act: -0.1700

Data Index: 4985554 (Open Web Text)

Max Activating Token Index: 1010

Text #7

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2181. Min Act: -0.1700

Data Index: 8480341 (Open Web Text)

Max Activating Token Index: 818

Text #8

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2079. Min Act: -0.1700

Data Index: 2060108 (Open Web Text)

Max Activating Token Index: 719

Text #9

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2066. Min Act: -0.1700

Data Index: 7160046 (Open Web Text)

Max Activating Token Index: 816

Text #10

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.1905. Min Act: -0.1700

Data Index: 4475556 (Open Web Text)

Max Activating Token Index: 974

Text #11

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2202. Min Act: -0.1700

Data Index: 6158144 (Open Web Text)

Max Activating Token Index: 860

Text #12

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.2023. Min Act: -0.1700

Data Index: 8621002 (Open Web Text)

Max Activating Token Index: 398

Text #13

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.1643. Min Act: -0.1700

Data Index: 1450381 (Open Web Text)

Max Activating Token Index: 576

Text #14

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.1588. Min Act: -0.1700

Data Index: 540972 (Open Web Text)

Max Activating Token Index: 646

Text #15

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.1906. Min Act: -0.1700

Data Index: 5842949 (Open Web Text)

Max Activating Token Index: 717

Text #16

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.1618. Min Act: -0.1700

Data Index: 7427336 (Open Web Text)

Max Activating Token Index: 427

Text #17

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.1907. Min Act: -0.1700

Data Index: 3958729 (Open Web Text)

Max Activating Token Index: 862

Text #18

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.1490. Min Act: -0.1700

Data Index: 8334470 (Open Web Text)

Max Activating Token Index: 938

Text #19

Max Range: 5.3728. Min Range: -5.3728

Max Act: 5.1623. Min Act: -0.1700

Data Index: 5286198 (Open Web Text)

Max Activating Token Index: 400