Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1465 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4425. Min Range: -5.4425

Max Act: 5.4425. Min Act: -0.1700

Data Index: 3422583 (Open Web Text)

Max Activating Token Index: 422

Text #1

Max Range: 5.4425. Min Range: -5.4425

Max Act: 5.2191. Min Act: -0.1700

Data Index: 3855981 (Open Web Text)

Max Activating Token Index: 342

Text #2

Max Range: 5.4425. Min Range: -5.4425

Max Act: 5.1206. Min Act: -0.1700

Data Index: 7235954 (Open Web Text)

Max Activating Token Index: 432

Text #3

Max Range: 5.4425. Min Range: -5.4425

Max Act: 5.1436. Min Act: -0.1700

Data Index: 4464766 (Open Web Text)

Max Activating Token Index: 1020

Text #4

Max Range: 5.4425. Min Range: -5.4425

Max Act: 5.1056. Min Act: -0.1700

Data Index: 5155670 (Open Web Text)

Max Activating Token Index: 292

Text #5

Max Range: 5.4425. Min Range: -5.4425

Max Act: 5.0413. Min Act: -0.1700

Data Index: 456779 (Open Web Text)

Max Activating Token Index: 879

Text #6

Max Range: 5.4425. Min Range: -5.4425

Max Act: 5.0893. Min Act: -0.1700

Data Index: 3656825 (Open Web Text)

Max Activating Token Index: 820

Text #7

Max Range: 5.4425. Min Range: -5.4425

Max Act: 5.0161. Min Act: -0.1700

Data Index: 3153064 (Open Web Text)

Max Activating Token Index: 595

Text #8

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9876. Min Act: -0.1700

Data Index: 4426413 (Open Web Text)

Max Activating Token Index: 973

Text #9

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9852. Min Act: -0.1700

Data Index: 5317589 (Open Web Text)

Max Activating Token Index: 802

Text #10

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9804. Min Act: -0.1700

Data Index: 5513611 (Open Web Text)

Max Activating Token Index: 402

Text #11

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9887. Min Act: -0.1700

Data Index: 3097568 (Open Web Text)

Max Activating Token Index: 266

Text #12

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9746. Min Act: -0.1700

Data Index: 4977832 (Open Web Text)

Max Activating Token Index: 44

Text #13

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9466. Min Act: -0.1700

Data Index: 608089 (Open Web Text)

Max Activating Token Index: 101

Text #14

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9627. Min Act: -0.1700

Data Index: 4645989 (Open Web Text)

Max Activating Token Index: 438

Text #15

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9200. Min Act: -0.1700

Data Index: 1290921 (Open Web Text)

Max Activating Token Index: 53

Text #16

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9063. Min Act: -0.1700

Data Index: 6472727 (Open Web Text)

Max Activating Token Index: 238

Text #17

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9516. Min Act: -0.1700

Data Index: 8407633 (Open Web Text)

Max Activating Token Index: 539

Text #18

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.9182. Min Act: -0.1700

Data Index: 4372457 (Open Web Text)

Max Activating Token Index: 215

Text #19

Max Range: 5.4425. Min Range: -5.4425

Max Act: 4.8645. Min Act: -0.1700

Data Index: 4392060 (Open Web Text)

Max Activating Token Index: 781