Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2711 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.5961. Min Act: -0.1700

Data Index: 6909835 (Open Web Text)

Max Activating Token Index: 695

Text #1

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.4375. Min Act: -0.1700

Data Index: 2399088 (Open Web Text)

Max Activating Token Index: 566

Text #2

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.4651. Min Act: -0.1700

Data Index: 5238855 (Open Web Text)

Max Activating Token Index: 624

Text #3

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.4959. Min Act: -0.1700

Data Index: 3214923 (Open Web Text)

Max Activating Token Index: 364

Text #4

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.4442. Min Act: -0.1700

Data Index: 5099811 (Open Web Text)

Max Activating Token Index: 419

Text #5

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.3966. Min Act: -0.1700

Data Index: 1451746 (Open Web Text)

Max Activating Token Index: 496

Text #6

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.3624. Min Act: -0.1700

Data Index: 8733036 (Open Web Text)

Max Activating Token Index: 717

Text #7

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.3680. Min Act: -0.1700

Data Index: 1451445 (Open Web Text)

Max Activating Token Index: 717

Text #8

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.2814. Min Act: -0.1700

Data Index: 5325341 (Open Web Text)

Max Activating Token Index: 463

Text #9

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.2976. Min Act: -0.1700

Data Index: 4986451 (Open Web Text)

Max Activating Token Index: 135

Text #10

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.2865. Min Act: -0.1700

Data Index: 7592533 (Open Web Text)

Max Activating Token Index: 285

Text #11

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.2007. Min Act: -0.1700

Data Index: 4814781 (Open Web Text)

Max Activating Token Index: 467

Text #12

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.1402. Min Act: -0.1700

Data Index: 5805848 (Open Web Text)

Max Activating Token Index: 1020

Text #13

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.1943. Min Act: -0.1700

Data Index: 6847062 (Open Web Text)

Max Activating Token Index: 423

Text #14

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.2523. Min Act: -0.1700

Data Index: 7393561 (Open Web Text)

Max Activating Token Index: 972

Text #15

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.2356. Min Act: -0.1700

Data Index: 7789560 (Open Web Text)

Max Activating Token Index: 193

Text #16

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.3047. Min Act: -0.1700

Data Index: 5449525 (Open Web Text)

Max Activating Token Index: 734

Text #17

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.1665. Min Act: -0.1700

Data Index: 1346259 (Open Web Text)

Max Activating Token Index: 897

Text #18

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.2391. Min Act: -0.1700

Data Index: 7381203 (Open Web Text)

Max Activating Token Index: 392

Text #19

Max Range: 5.5961. Min Range: -5.5961

Max Act: 5.1784. Min Act: -0.1700

Data Index: 8621862 (Open Web Text)

Max Activating Token Index: 713